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ABSTRACT 

This manual presents information on three instruments 
designed for surveying mental health problems in schools* Part I 
describes the instruments, vhich are two questionnaires called the 
School Problem Area Survey — Staff, and the School Problem Area 
Survey — Students, and a guide to b^ usled in interviews, called the 
Demographic Information Form as well as procedures for their use. The 
questionnaires are designed to obtain information about potential 
problems in schools which may be amenable to treatment by mental 
health indirect services program carried out by mental health 
consultants and members of the schools" staff* The guide should help 
consultants and principals gain a comprehensive picture of their 
schools which would aid them in problem solving. Part I, in addition, 
contains outlines on the necessary steps to be taken to obtain the 
interview and questionnaire data, statistical pirocedur^s involved in 
<1ata analysis and guidelines for the assigning of priorities for the 
interpretation of data from three sources. Part II is an exploration 
of the general meanings of reliability and validity^ It is concluded 
that measures of internal consistency are the most appropriate 
measures of reliability. Face validity and demonstrations of the 
utility of the instruments seem most appropriate in addressing the 
issue of validity. (Author/BJG) 
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FOREWORD 



A persistent question raised about education today concerns ways to make 
schooling a more satisfying experience for students and staff alike. The instroments 
described in this manual have been designed to obtain the perceptions of the 
students, the staff, and the principal of a school about a variety of common school 
problems. As diagnostic tools, these instruments have proved useful to mental health 
consultants to schools and to school personnel in identifying problem areas that 
warrant corrective action. 

Three instruments for gathering data have been developed. Two are 
self-administered questionnaires to be completed by school staff and students and 
the third serves as an interview guide for use with school principals. The instruments 
were developed and tested in 50 junior high schools in six states by the Human 
Resources Research Organization under a grant from the National Institut3 of 
Mental Health. 

We are indebted to Dr. Charles Windle, Program Evaluation Specialist, Division 
of Mental Health Service Programs, National Institute of Mental Health, and to 
Dr. Beryce MacLennan, Director, Mental Health Study Center, National Institute of 
Mental Health, Adelphi, Maryland. As Project Officer, Dr. Windle provided guidance 
and assistance during the progress of this research. Dr. MacLennan was a consultant 
to the project and an observer at a dissemination and training workshop in which the 
procedures and instruments were reviewed. 

Other project consultants, who have generously contributed time, valuable 
criticism, and assistance, were Dr. Ira Iscoe, Director, Counseling-Psychological 
Services Center, University of Texas at Austin; I>r.A. Russell Lee, Director, 
Emanuel Medical Center, Turlock, California; Dr. Charles D. Spielberger, Professor 
and Director, Doctoral Program in Clinical and Community Psychology', Department 
of Psychology, University of South Florida at Tampa; and Mr. Mitchell Baris, 
research psychologist, Adams County Mental Health Center, Commerce 
City, Colorado. 

Mr. Dave Norman, of the U.S. Naval Postgraduate School, Monterey, 
California, provided valuable assistance in data processing. 

Special appreciation is expressed to the more than 50 school principals who 
cooperated in the study and made possible the collection of data in their schools. 

The study was conducted at HumRRO Western Division, Carmel, California; 
Dr. Howard H. McFann is the Division Director. The research team consisted of 
Dr. Elaine N. Taylor, Principal Investigator, Dr. Robert Vineberg, and Dr. S. James 
Goffard. Significant contributions were made in data collection by Mrs. Dorothy 
Herbert and Mr. Alton Boyd and in data analysis by Mr. Terrence McGiveran. 

The research was performed under NIMH Grants MH 21708 01, -02, and -02S1 
during the period ljune 1972 to lJuly 1974. The manual is designated as 
HumRRO Technical Report 74-22, October 1974. 



The effort reflected in this manual was directed ioward helping define problem 
areas in schools, as a necessary first step in planning move effective school programs 
and mental health consultation services to schools. It is our hope that these 
instruments and procedures will prove to be of continuing value to personnel 
responsible for administering school programs and to mental health workers who 
consult with them in problem-solving endeavors. 



Meredith P. Crawford 
President 

Human Resources Research Organization 



September 1974 



This manual presents infonnation on three instruments designed for 
surveying mental health problems in schools. Part I describes the instruments, and 
procedures for their use. Part II contains more technical information on the 
formal characteristics of the instruments. 

The instrmnents are two questionnaires (the School Problem Area 
Survey-Staff and the School Problem Area Sxirvey— Students), and a guide (the 
Demographic Information Form) for an interview by a mental health consultant 
with a school principal. 

The questionnaires are designed to obtain information about potential 
problems in schools which may be amenable to treatment by mental health 
inclirect services programs carried out by mental health consultants and members 
of the schoors staff. The questionnaires are also appropriate for use by school 
personnel working independently to identify problem areas in their school. 
Reading level of the student questionnaire is estimated to be high sixth grade. 
Time to complete the questionnaires is approximately 20 minutes. 

The interview form is designed to provide the consultant with a 
comprehensive description of the school and to guide the principal through a 
systematic exploration of those features of the school and its setting which may 
give rise to problems or act as constraints upon possible solutions to problems. 

Part I also contains (a) an outline of the necessary steps to be taken to 
obtain the interview and questionnaire data, (b) an outline of the statistical 
procedures involved in data analysis, and (c) a set of guidelines for interpreting 
the data firom the three soi\rces of information and for assigning priorities 
to problems. 

The general meanings of reliability and validity, as these concepts are 
applicable to the instruments, are explored in Part II. . It is concluded that 
measures of internal consistency are the most iqppropriate measures of reliability. 
Face validity and demonstrations of the utility of the instruments seem most 
Impropriate in addressing the issue of validity. It appears that the questionnaire 
can produce data with high reliability (for example, rs of .95 for staff and .82 for 
students in one school). They are capable of discriminating among problem areas 
within schools and do differentiate between schools. Their face validity 
seems good. 

Appendix materials include (a) directions for hand computation of the data, 
(b) keypunch instructions and a computer program, and (c) notes on development 
of the instruments. 

Available on order with the Instruments are a form for summarizing staff and 
student responses to the questionnaires and a chart for preparing a profile of staff 
and student responses to items grouped into problem areas. 
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One responsibility of Community Mental Health Centers is to provide 
indirect services to schools.' These services have varied considerably, but they can 
be grouped into three main types: 

• Case- and/or Client-Centered Staff Development Consultation: 
Assisting a school ctaff in the prevention, control, or treatment of 
mental and emotional problems of individual students. 

• Agency 'Centered Staff Development: Helping a school staff solve the 
more general or systemic problems of the school which may be 
detracting from the efficient accomplishment of its overall mission. 

• Project Development: Aiding in developing special programs aimed at 
groups of students at risk— for example, potential dropouts or 
drug abusers.^ 

Typically, these services have been undertaken without a systematic study of 
the needs of the school. Consequently, they have tended to be unplanned, 
unprogrammatic, and unresponsive to the immediate needs of most of the 
members of the school staff. ^ Too often, actions seem to be determined by the 
consultant's theoretical predilections, by a desire to cling to familiar modes of 
intervention, or by the practical necessity of intervening in a nonthreatening way, 
rather than by a systematic analysis of the needs of the school. 

A comprehensive program of relevant indirect services can be provided to a 
school only when the plans for intervention are based on: 

• Comprehensive information about the school. 

• Systematic procedures for estimating the relative salience of the 
various problems identified in the school. 

• An appropriate rationale for assigning priorities for intervention 
among such problems. 

'PL 88-164 et seq. 

^For a more complete description of these types, see P^ontague, Ernest K. and Taylor, 
Elaine N., Preliminary Handbook on Procedures for Evaluating Mental Health Indirect Service 
Programs in Schools, HumRRO Technical Report 71-18, August 1971. 

^McClung, Franklin B. and Stunden, Alastair H,, Mental Health Consultation to Programs 
for Children, Public Health Service Publication No. 2066, National Institute of Mental Health, 
Bethesda,Md., 1970. 
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Procedures foi Surveying School Problems 



The Project 

For use by Community Mented Health representatives in planning mental 
health consultations and other indirect services to schools, the Human Resources 
Research Organization has developed a cluster of materials, which includes three 
survey instruments and this manual to assist users of the survey instruments. 

These instruments, which gather information from school staffs and students, 
are screening devices for identifying possible problem areas that should be con- 
sidered in planning a program of indirect services. 

The three screening instruments are. 

• The School Demographic Information Form 

• The School Problem Area Survey for School Staff Members 

• The School Problem Area Survey for Students 

The School Demographic Information Form is to be filled out by a con- 
sultant during an interview with the principal of a school. It covers information 
about five topics: 

• General Characteristics of the school and the community in which it is 
located. 

• Extrinsic Factors that may affect the school operation. 

• Specific School Characteristics, including policies, curriculum and 
programs, and special problems. 

• Staff Characteristics. 

• Summarization by the principal of the schooPs most pressing needs 
and greatest strengths. 

The School Problem Area Sun'ey instruments are two questionnaires, one to 
be administered to staff and one to students, to explore their perceptions about 
the characteristics of, and the interrelationships among, the school admi >tration 
(primarily the principal), the teachers, the students, and the community. 

Also included in the cluster of materials developed for the survey are two 
forms-t)ne for use in summarizing survey information on problem areas and 
p orities, and another for use in drawing a school profile of problem areas. 



The Manual 



This manual has been written for the convenience of the readers v^ho will 
use the survey instruments. There are two sections: Part I provides a statement of 
the various purposes of the instruments, their description, and the procedures to 
be foUowed in assessing school needs; Part II presents information on the reli- 
ability and validity of the question nairesHjut still at a general level. Included with 
Part II, but as a separate section for readers interested in more technical informa- 
tion, are Notes supporting the general statements about reliability and validity. 

Specifically, the manual provides: 

(1) Syrtematic procedures for gathering coniprehensive information 
about a school. These procedures give the consultant a broad range of data on 



lERIC 



Introduction 



(a) the background and composition of the school; (b) sources of problems within 
the school; (c)loci of disagreement and friction among the administrators, 
teachers, students, and community. 

(2) A set of procedures for analyzing the survey data to yield estimates 
of the relative salience of the various problems in the school These procedures 
for comparing and combining information allow the c o order the 
problems within a school. 

(3) Guidelines for assigning priorities for intervencion in the various, 
and usually quite different, problems of a school. When the most salient problems 
have been identified, they are rank ordered in terms of the extent to which they 
disrupt achievement of the educational goals of the school. 
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Fart I 

THE INSTRUMENTS AKO 
THEIR APFLieATION 



Uset of the Instruments 

These survey instruments can be used in a variety of ways: 

• A mental health consultant and z school principal can make 
systematic plans for utilizing the indirect services of a 
msntdl health facility in a school setting. 

• A school principal and his or her staff can assess the needs 
of their school and can either deal directly with the 
problems identified as salient, seek aid using resources of 
the school system, or caU upon outside consultants to assist 
in solving the problems. 

• A school superintendent, working alone or vrith a mental 
health consultant, can assess the problems of the various 
schools in the district and thus make more effective plans 
for allocating the resources available. 

• Using data from several schools, a mental health facilitj" can 
select those schools and problems where its resources can 
be most appropriately used. 

In addition, the survey instruments can: 

• Provide baseline data for comparison with "post-treatment" 
data, thus giving an estimate of the effects of consultation 
programs or other varieties of indirect services. 

• Provide datp to document requests for the funding of 
indirect services. 

• Provide material for training programs for mental health 
consultants. 
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Procedures for Surveying School Problems 



• Serve as models for assessing needs and planning 
interventions with agencies and institutions other 
than schools. 

These instruments and the associated procedures will serve additional 
purposes. The interview with the principal in which the School Demographic 
Information Form is filled out provides a means for establishing rapport and 
credibility during the early phases of consultation. Also, the data obtained from 
the survey instruments, when reviewed with the principal and members of the 
staff, serve an educational function by introducing different points of view and 
identifying problems not previously recognized. Such a review of the data helps 
establish tiie consultation process as a joint problem-solving endeavor. 



The School Pomografhie Information Form 



The School Demographic Information Form is a guide which the consultant 
will use in an interview and discussion with the principal of a school. Depending 
upon the amount of discussion and amplification of various topics, the interview 
takes from one to c^wo hours to complete. The content of the interview is based 
largely on the experiences of many peisons who have worked in schools (school 
personnel, mental health consultants, and educational research workers). The 
Information Form provides for a systematic exploration of those features of a 
school and its setting which may give rise to problems in the school or which may 
act as constraints upon possible solutions to problems. Extracurricular programs, 
for example, may not be feasible in a school where most of the students have to 
be bused in; strong community opposition to bond issues may make it all but 
impossible to improve a school's facilities.^ 

Usually, this interview will be part of the "entry phase'* of the consul- 
tation process.^ 

This interview serves several purposes, in addition to giving the consultant an 
opportunity to gain rapport with the principal and establish credibility as 
a consultant: 

• It can give the principal an understanding of the 
characteristics and possible range of indirect services 
available 

• It presents the principal with an opportunity to sanction 
(or not to sanction) further consultation. 

' The form h&s he. n revised and reorganized several times. See Appendix C of this manual 
for some of the analyses which were carried out on pilot data to refme the interview form. 

^For a detailed discussion of this **ehtry phase,*' see A. Beisser and R. Green, Mental Health 
Consultation and Education, National Press Books » Palo Alto, California, 1972. 

15 




The Instruments 



• It encourages the principal to share knowledge of the school 
with the consultant. 

• Through their joint analysis of the school's problems, it 
allows the principal and the consultant to formulate an 
initial list of isoues for consultation. 

• It g i ves the consultant an opportunity to make 
arrangements fcr administering the questionnaires to the 
staff and students. 

• It encourages the consultant to consider the nature of the 
school and its setting. (Experience has shown that mental 
health consultants who have been involved solely in client- 
centered consultation often know little about the schools 
where they have been consulting. Consequently, they are 
not in a position to go beyond client-centered consummation 
in the services they can offer the school.) 

The School Demographic Information Form, arranged in five sections, is 
presented on pages 10-19. 
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School District 



Mental Health Facility. 



SdiocI 

PrI nctpal 

Consultant^ 

Years as principal. 

This school Other schools pj^l^j 

Years in teachirig and administration- 

This school Other schools Date 

Responsible person in principal's absence 



School Demographic 
Information Form 



HuMRRO W«FO«M t 



1874 

©Human Resources Research Organization (HumRRO) 
300 Nor^h Washington Street 
Alexandria, Virginia 22314 

These materials were developed under a grant torn the National Institute 
Mental Health, US. Departnf>ent of Health, Education, and Welfare. 
Reproduction in whole or m part is permitted for any purpose of the 
US, Government. 
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I. GENERAL CHARACTERISTICS 



A. Grades served (Underline appropriate set): 
K123456789 10 11 12 

B Length of time this school has been in operation^ 

Less than 1 year 

1-2 years 

3-5 years 

6-10 years 

More than 10 years 

C. Number of students enrolled: 



G. 



0. Average daily attendance: 



E. Student/Teacher (Professional) ratio: 



F. School location and source of students: 



Location 
(check those applicable) 

Metropolitan (IbO^OOO or more) 

Inner city 

Not inner city 

Big city (50.000 • 150.000) 

Small city (10.000-50.000) 

Town or rural (Lesj than 

10,000. or in country) 



Percent of 
students from: 



Percent of students who: 

1. Can walk to school 

2. Need transportation 



% 
% 
% 
% 

% 



% 
% 



Approximate percent of students from each 
socio-economic level: 



Lower Low 
Upper Low 
Lower Middle 
Upper Middle 
Upper 



% 
% 
% 
% 
% 



M. 



0. 



Approximate percent of student families m the 
following income leve!s: 



Less than $4,000 
4,001 - 8,000 
8,001 - 12,000 
12,001 - 16,000 
More than 16,000 



% 
% 
% 
% 
% 



Ethnic composition of student body 
(Approximate numbers): 

American Indian 

Black 

Oriental 

Spanish surnamed 

White 

Other 



Number of students whose families are on 
welfare: 

Number of students on free lunch 
program: 

Estimated number of students enrolled in the 
following schools who are eligible to attend 
this school: 

a. Private 

b. Parochial 

c. Alternative 

d. Specialized 



(e.g., vocational, art, music, etc.) 

Total number of schools in this school 
district: 



Total number of schools in the district serving 
approximately the same grade levels as this 
school: 
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!!. EXTRINSIC FACTORS 



A. is the community surrounding the school in the 
process of rapid social change? (Check applicable 
statements and describe.) 

1. Change in socio-economic composition 

2. Change In ethnic composittori 

3. Change in predominant character 
(agricultural, residential, industrial) 

4. Other changes: 

Description : 



2. Toward new programs and/or curriculum, 
generally (check ont): 

Obstructive 

Supportive 

About evenly split 

Indifferent 

D. Are there special factions or pressure groups, 
either beneficial or disruptive, in this com- 
munity which have an impact upon principal, 
faculty, or students? 

1. No Yes 

2. If "Yes," what are these groups? 



B. Community financial suppott for schools: 

1. Financial support is (check one): 

Minimal 

Adequate 

Generous 

2. Per pupil expenditure: 

In this school $ 

In this school district $ 

In the state $ 

3. Attitudes toward special tax assessments, 
such as a tax override (check one): 

Negative 

Positive 

About evenly split 

Indifferent 

C. Community attitudes: 

1. Toward students, generally (check one): 

Hostile 

Friendly 

About evenly split 

Indifferent 



3. How do these groups affect the principal, the 
faculty, or the students? 

a. Beneficial 



b. Disruptive 



E. Does this school have any problems with the 
surrounding neighborhood? 

1. No Yes 

2. If "yes," what are they? 
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III. SPECIFIC SCHOOL CHARACTERISTICS 



A. Orientation of school— Emphasis upon various curriculum fields in relation to student needs: 

Curriculum Under- About Over- 

Fields emphasized right emphasized 



1. Academic 

2. Vocational 

3. Business 

4. Other 

5. Other 



B. Amount of pressure for student achievement in this school: 

Too much About right Too little 

From principal and 

administrative staff 

From teachers 

From parents 

From students 



C. Percent of student body in the following levels of achievement relative to national norms: 
Below average Average Above average Test used 



Reading % % % 

Arithmetic % % % 



D. Freedom of choice for students from among: 

None Some A lot 



Standard courses needed to complete graduation 

requirements 

Electives 

Activities (e.g., interest groups) 

Organized athletics 

E. Student participation In and support of extracurricular activities: 
Low Moderate High 

F. Additional courses or special sections of regular courses for exceptional children: 
Fast learners: No Yes 



Slow learners: No 



Yes 



Special instructional progtams carried either as separate courses or as units of other courses: 



Perceived community 
attitude: 
Favorable + 

Currently Not covered Neutral 0 

covered Not needed Needed Unfavorable ^ 



1. Family living 

2. Sex education 

3. Drug education 

4. Vocational training 

5. Ethnic studies 

6. Other 

7. Other 



Groups that might profit from special mental health assistance: 



In Column A, estimate number of students you expect to have this year for each problem listed. 



In Column B, estimate the number of students you expect to participate in any program your 
school may have (alone or in collaboration with another agency) which deals with that problem. 



Column A Column 6 

Number of students Number of students 
with problems expected In program 



1. Special Education 

a. Blind or partially sighted 

b. Deaf or hard of hearing 

c. Educable mentally retarded 

d. Educationally or emotionally handicapped 

e. Mentally gifted 

f. Other 

2. Probation from courts 

3. Suspension from school 

4. Drug abuse 

5. Venereal disease 

6. Pregnancy 

7. Teen-age parents 

8. Underachievers 

9. Students who are one or more years behind 

10. Dropouts 

11. Potential dropouts 

12. Habitual absentees 

13. Student turnover (number transferring in) 

14. Students who speak non-standard English -that is, those 
with regional or ethnic dialects not, common to 

the faculty 
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Disruptive incidents during the last school year (estimate number): 

1. Petty theft • items under $10 

2. Major theft 

3. Assault • causing absence of at least 1/2 day 

4. Extortion 

5. Acciderits or injuries at school requiring more than simple first aid 

6. Other 

Cost of vandalism to school during last year: $ 

DiscipHne: 

1. Disciplinary problems referred to special school personnel: 

Infraction Cases per 

Failure to perform school work 

Chronic absenteeism 

Disrespect to people in authority 

Disorderly behavior 

Destruction or stealing of property 

Assaultive behavior 

Drug offenses 

Sex offenses 

Other 

Other 

Total 

2. Time spent on disciplinary cases by persons within school: 

Approximate hours 

By whom per month 

Principal 

Assistant principalis) 

Dean(s) 

Other 

Other 

3. Referrpis to persons or agencies outside of school: 

Person or Agency Cases per month 

Parents or legal guardian 

Pupil personnel services of district office 

Superintendent or assistant superintendent 

School board 

Juvenile authority (probation department or court) 

Police department 

Community counselor 

Physician 

Psychiatrist 

Clergy 

Other . 

22 



4. Corporal punishment: 

a. Is corporal punishment permissible? 

b. Is corporal punishment used? 

Of not used, disregard remainder of 4) 

c. How many times per month is corporal punishment used? 

d. Who may administer corporal punishment? 
(Check all that are applicable) 



No. 
No. 



Principal 

Assistant principalis) 

Dean(s) 

Teachers 

Aides 

Parents at school . . . 

Other 

Other 



Yes. 
Yes- 



Dress Code: 

1. Does this school specify any standards for acceptable dress, other 
than no bare feet? (if 'Wo/' disregard remainder of L) Yes_ 



2. Are these standards embodied in a written dress code? 

3. What do these standard: specify? (Check all items that 
are applicable.) 



Yes_ 



No bare midriffs . . . . 
Belts to be worn with 

trousers 

No short-shorts . . . . 

Skirt length 

Male hair length . . . . 



No 
No 



No beards or mustaches . . . 
No tight sweaters on females. 
No trousers oh females 

No outlandish dress 

Other 

Other 



Who was involved in setting these standards? 
(Check all items that are applicable.) 

School board 

Central administration 

Administration of this 

school 



Teachers. 
Parents . 
Students 
Others 



5. Are the students well-informed about these standards? 
No Yes 



How do you assess these standards? 
Conservative Modest _ 



Liberal . 



How strictly are these standards enforced? 
Very strictly Not very strictly. 



Not at all 



8. What is the student body reaction to these standards? 
(Indicate percentages) 



Favorable 
Passive acceptance 
Rebellious 



% 
% 
% 
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IV. STAFF CHARACTERISTICS 



A. Qualifications for hiring teaching staff: 

Requirements 

A. A. or A.S , 

B. A. or 8.S 

Graduate work 

Teaching credentials 

Teaching experience 

Other 

B. Size of staff: 



Yes 



No 



"Ye;," Is it evar waived? 
Yes No 



Number 



Is number 
suffic ient? 

Yes 



No 



Assistant principals 

Deans 

Curriculum specialists 

Classroom teachers 

Teaching specialists for educationally or emotionally 

handicapped students 

Teaching specialists for educable mentally retarded students. 

Speech and hearing specialists/therapists 

Academic counselors 

Vocational counselors , 

Personal counselors 

School nurses 

School psychologists 

librarians , ■ • 

Cafeteria staff 

Secretarial staff 

Paraprofesslonals 

Clerical assistants 

Library aides 

Other aides 

Custodian; 

Security guards 

Other 

Other 



C. Teaching experience of classroom teachers: 



Years In this school 



Number of 
Teachers 



Total years of 
teaching experience 



Number of 
Teachers 



First year 

1 -2 years 

34 years 

5-6 years 

More than 6 years 



First year 

1-2 years 

34 years 

5-6 years 

More than 6 years 
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D. Ethnic composition of professional staff (indicate number): 



American Indian Spanish Surnamed 

Black White 

Oriental Other 

E. Teachers' Union and Negotiating Committees: 

1. Do the teachers belong to a union? 
No Yes 

2. Is there a teachers' negotiating committee for dealing with the school board^ 
No Yes 

3. For this building, is there a teachers' negotiating committee for dealing with the principal^ 
No Yes 

F. Inservlce training: 

1. Is there provision for Inservice training in this school? 
No Yes 

2. What IS the approximate numbei* of hours of Inservice training available to your staff this year 
from the following sources^ 

Hours 

Personnel within this school 

Personnel with'n this district 

Consultants hired by the district 

Other 

3. Is release time generally arranged? 
No Yes 

Explain, if necessary: 



4. Are most inservice training programs open to all the professional staff? 
No Yes 

Explain, if necessary: 



G. Turnover of professional staff in this school during the last year: 

Reason for turnover Number Involved 

Transferring within district 

Transferring to another district 

Going on leave 

Taking graduate work 

Retiring 

Other 
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A. 



V. SUMMARIZATION 



In summary, what do you see as the most pressing needs and problems of this school? 



Assign a number to each of the above from most (#1) to least important. 
B. What do you see as the greatest strengths of this school? 



Assign a number to each of the above from most (#1) to least important. 
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Procedures for Surveying School Problems 



The Questionnaires 



A CoM^Mii^i for eUssifyiiis Sekool Froblwis 

The questionnaires are used to obtain the perceptions of staff and students 
about problems in their school. The content of the questionnaires is based upon a 
conceptualization of a school as a moderately autonomous subsystem of a 
community, a subsystem consisting of three primary cort^ponents: the students, 
the teachers, and the principal or administration. Problems in a school can arise 
from seven main sources, either from within one of these three components, or at 
one of the three interfaces among the components or between the community 
and the school. 

Each of the components may be itself a relatvely complex system. Problems 
may arise, therefore, at particular parts of an interface betwee :i two components. 
For example, the teachers mav be dissatisfied with the principal as an adminis- 
trator but quite pleased with him or her as an educator or as a leader or simply as 
a person to work with. 

In constructing the questionnaires, a pool of potential items was developed 
following (a) extensive reading in the educational, psychological, and sociological 
literature lelated to schools; (b) interviews with a numbet of mental health, 
educational, aiid sociological consultants to schools; and (c) interviews with school 
principals, :xAioo\ psychologists, and school teachers.' 



Tlie ProbliM /lri« Purvey for $ehool Sfoff Menbort 



The "School Problem Area Survey: Staff" is a questionnaire m which 
teachers and other school staff estimate the salience in their school of a variety of 
potential problems. There are 70 items in the questionnaire, which requires 
approximately 20 minutes to complete. 

The questionnaire {see pages 22-25) deals with two types of information. In 
the first half (six sections of six questions each^ ), the staff members estimate the 
severity of the problems they see in the characteristics of, and interrelationships 
among, students, teachers, the principal, and the community. In the remainder of 
the questionnaire they rate the severity of a variety of problems associated with 
the students, teachers, administrator, facility, aiio community. 

^The questionnaires have gone through a series of major revisions. See Part II of this manual 
for a discussion of the formal characteristics of these questionnaires and Appendix C for a review 
of the developmental history of the questionnaires. 

^See Table 1 for further details. 
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The Instruments 



The respondents are asked to consider whether they perceive a particular 
item as an "Extreme Problem," "Considerable Problem," "Moderate Problem," 
"Little Problem," or "No Problem At All" in their school. An "Exc^tionally 
Good" response column is also provided. While this latter response is treated 
statistically as if it -.vere "No Problem At All," it has been included to provide the 
principal, consultant, and others with supplementary information that may be 
useful in interpreting the data. 

The items in this questionnaire are intended simply to screen for problems. 
For example, unfavorable answers to Item 8— *The respect teachers and students 
have for one another"-^ould indicate that a problem exists, but the item does 
not specify the precise nature of the problem. Several additional items would be 
needed to specify in detail the nature of the problem. To keep the questionnaire 
as brief as possible, most items have been written lu this general form. Once the 
problems have been identified generally, their specific nature can be elaborated in 
meetings of the consultant with the principal and the staff. 



zs 



21 



School Problem Area Survey: 

Staff 



What This Is About 



Every school has its strengths and its weaknesses. What are they in 
your school? The purpose of this questionnaire is to give you a chance to 
identify some of the strengths and weaknesses that you see in your school. 

Obviously, we cannot list all possible strengths and weaknesses. The 
best we can do is list a number of general instances of each and hope that 
our list will be systematic and comprehensive enough to cover most of the 
specific strengths and weaknesses that you ses in your school. 

Please look at each of the items belcvv and decide whether you 
believe the situation referred to is an Extreme Problem, a Considerable 
Problem, a Moderate Problem, a Little Problem, No Problem At All, or, 
in fact, Exceptionally Good in your school, and then put an X in the 
appropriate co*umn. Even though you feel uncert2un about some of the 
items, report your opmion, your feelings, or your impressions to the best 
of your knowledge. The consensus on these items would surprise you. 
Since it is the consensus and not individual responses that we want, do 
not sign your questionnaire. The questionnaires are and will remain 
anonymous. 

Record your first quick response to thes, items without pondering 
over them. Please use the final page of this form to write any further 
comments you may have. 
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IS THIS A PROBLEM IN YOUR SCHOOL? 
(Put an X in the appropriate column.) 



1. 


The way the students get along with one another 


1 


2 


3 


4 


5 


6 


2. 


The number of students who don't like going to school and don't 
do their schooi work 




2 


3 


4 


5 


6 


*j* 


StiiHpPtc from ethnic minorities 




2 


3 


4 


5 


6 


4. 


Friction or hostilltv between groups of students 




2 


3 


4 


5 


6 


5. 


Capat)le stud&its who feel that going to school is pretty much a 
waste of time 




2 


3 


4 


5 


6 


6. 


The number of students who don't seem to do much with other 
students-who are "'oners" 




2 


3 


4 


5 


6 


7. 


The usual social atmosphere or feeling in the classroom 


■ 


2 


3 


4 


5 


6 


3. 


The respect teachers and students have for one another 




2 


3 


4 


5 


6 


9. 


Teachers who don*t seem to care about the personal and 
educational problems of their students 


• 


2 


3 


4 


5 


6 


10. 


Teachers who put too much pressure on their students to get good 
grades 


■ 


2 


3 


4 


5 


6 


11. 


Unfair treatment of students by teachers 




2 


3 


4 


5 


6 


12. 


Teachers who won't admit making mistakes or think there is only one 
right answer to every question 




2 


3 


4 


5 


6 


13. 


Teachers who complain about other teachers 


- 


2 


3 


4 


5 


6 


14. 


Disagreements among the staff on the proper educational goals 
for the school 




2 


3 


4 


5 


6 


15. 


Disagreements among the staff on the proper balance between 
traditional and innovative approaches to teaching 




2 


3 


4 


5 


6 


16. 


Communication among the school staff 




2 


3 


4 


5 


6 


17. 


Teachers who seem bored with teaching 




2 


3 


4 


5 


6 


IB. 


Older teachers vvho are reluctant to accept newer teachers 
as colleagues 




2 


3 


4 


5 


6 
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IS WIS A PROBLEM IN YOUR SCHOOL? 
(Put an X in the appropriate column.) 




19. The way the principal gets along with the students 

20. A feeling in the school that conform ity and orderliness among 
the students are more important than freedom and individuality 

21 . Loose or lax policies on student behavior which foster disorder- 
liness and disorganization 

22. Absence of a schoolwide system for identifying and dealing with 
students who have special educational needs or problems 

23. The amount of influence student opinion has on tne way the school 
is run 

24. The way students are assigned to classes, graded and promoted 

25. The way the principal gets along with the teachers 

26. The way the principal handles staff conflicts 

27. The amount of teachers' time taken up hy non-teaching activities 

28. Criticism by the school administration of teachers who do not 
maintain tight control over their students 

29. Understanding how the principal evaluates teaching performance 

30. Disagreements between the principal aiuJ the teachers on educational 
matters 

31 . The way the teachers get along with parents 

32. The way the people in this nelghbo'^hood feel about the school 

33. Lack of community interest in the schools 

34. Teacher dissatisfaction with the cor^munity 

35. Community dissatisfaction wit the schools 

36. School policies that conflict with pareritr.' ideas 
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IS THIS A PROBLEM IN VOUR SCHOOL? 
(Put an X in the appropriate column ) 




1 


•> 


3 


4 


5 




6 




2 


3 


4 


5 




6 




2 


3 


4 


5 




6 




2 


3 


4 


5 




6 




2 


3 


4 


5 




6 




2 


3 


4 


5 




6 




2 


3 


4 


5 




6 




2 


3 


4 


5 




6 




2 


3 


4 


5 




6 




2 


3 


4 


5 




6 




2 


3 


4 


5 




6 




2 


3 


4 


5 




6 




2 


3 


4 


5 




6 




2 


3 


4 


5 




6 






2 


3 


4 


5 




6 




2 


3 


4 


5 




6 




2 


3 


4 


5 




6 




2 


3 


4 


5 




6 




2 


3 


4 


5 




6 




2 


3 


4 


5 




6 




2 


3 


4 


5 




6 




2 


3 


4 


5 




6 




2 


3 


4 


5 




6 






2 


3 


4 


5 




6 




2 


3 


4 


5 




6 




2 


3 


4 


5 




6 




2 


3 


4 


5 




6 




2 


3 


4 


5 




6 




2 


3 


4 


5 




6 




2 


3 


4 


5 




6 






2 


3 


4 


5 




6 




2 


3 


4 


5 




6 






2 


3 


4 


5 




6 




2 


3 


4 


5 




6 




1 


2 


3 


4 


5 




6 



37. 
38. 
39. 
40. 
41. 
42. 
43. 
44. 
4b. 
46. 
47. 
48. 
49. 
50. 

51. 
52. 
53. 
54. 
55. 
56 
57. 
58. 
59. 

60. 
61. 
62. 
63. 
64. 
65. 
66. 

67. 
68. 



Underachievement 

Cheating 

Cutting class 

Absenteeism 

Dropout rate 

Vandalism 

Theft 

Drugs , 

Alcohol 

Sexual promiscuity 

Teen-age parents 

Delinquency 

Profanity 

Violence or threats of violence 



Transient students 

Students who speak non-standard English 

Ethnic tensions 

Student poverty 

Student health 

Changing neighborhood characteristics 

Divisive community influences 

Busing to improve racial balance .... 
Changing composition of student body 



Discipline 

Dress code 

Irrelevant curriculum 

Inadequate programs for gifted students 

InadiK^uate remedial services 

Inadequate counseling services 

Inadequate medical services 



The condition of the building and/or the grounds 
Class size . 



69. Teacher turnover . . . 

70. Teacher absenteeism 

71. Other 
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Procedures for Surveying School Problems 



The Problem Area Survey for Studenfs 



The "School Problem Area Survey: Students" (pages 27-29) is very much 
like the staff questionnaire in form and content. The students, like the teachers, 
estimate the salience of a variety of potential problems in their school. This 
questionnaire, which has 49 items and requires about 20 minutes to complete, is 
most appropriately used at intermediate and high school levels. While it is 
readable at the sixth grade level, the meaningfulness of the responses is 
questionable at that level and below.' 

For efficiency, it should be given to a sample of students rather than to an 
entire student body. Guidelines for sampling are presented in "Introducing the 
Instruments Into a School and Obtaining Data," which appears later in this 
manual. The student questionnaire should be administered as soon as possible 
after the staff questionnaire has been administered. 



Comparability of Staff and Student Questionnaires 



A tabulation of items by problem area fur each questionnaire is presented in 
Table 1, indicating which items are common to the staff and student 
questionnaires and which are unique. Items have no counterpart in the other 
questionnaire if (a) the respondent group could not reasonably be expected to be 
well-informed on the subject, and (b) the item might be considered inappropriate 
enough in some schools to preclude use of the questionnaire. 



ERIC 



26 



^ An estimate of the required reading level of the instructions is at high sixth grade, based on 
the Flesch count, using word and sentence length. 
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School Problem Area Survey: 
Students 



What This Is About 

Every school has its own strong points and weak points. What do you 
think are the particular strong points and weak points of your school? The 
purpose of this questionnaire is to give you a chance to point out some 
of them. 

It is not practical to list everything that anyone ever thought was a 
problem in a school. The best we can do is list some of the general kinds 
of problems that students see in their schools. What we have here is a list 
that will apply to most students in most schools. We hope that each of 
the particular problems that you see in your school will be covered by one 
of the more general problems that we have listed. 

For example, one of the items is: 

"The usual social atmosphere in the classrooms." 

The "social atmosphere" in a classroom is no problem when students 
and teacher are all working together toward the same goals, when there is 
a ^ree, open and positive feeling in the room. The social atmosphere in a 
classroom becomes a problem when there are lots of strong negative 
feelings— anger, hostility, anxiety, frustration— that make it hard to get any 
work done. It is up to you to decide whether you feel that the usual 
social atmosphere in your own classrooms is an Extreme Problem, a 
Considerabte Problem, a Moderate Problem, a Little Problem, No Problem 
At All^ or Exceptionally Good. When you have decided, put an X in the 
box under your choice and go on to the next item. 

Please do not sign this questionnaire. We are interested in how groups 
of students feel, not in identifying the feelings of individual students. 

Read each item carefully, but don't think too long about your 
answer. Give your first quick reaction and go on to the next item. Please 
use the final pag' ^ this form to write any further comments you 
may have. 

HumRRO W.roRM 3 
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IS THIS A PROBLEM IN YOUR SCHOOL? 
(Put an X for each problem m the box 
under your choice.) 




1 . Not enough school subjects to choose from 

2. Not enough extracurricular activities 

3. Getting the stuuents to show some school spirit 

4. Too much noise and confusion 

5. A generally unfriendly atmosphere 

6. Theway this school is run 

7. The way the students get along with one another 

8. The number of students who don't like going to school and don't 
do their school work 

9. Students from ethnic minorities 

10. Friction or hostility between groups of students 

11. Capable students who feel that going to school is pretty much a 
waste of time 

1 2. The number of students who don't seem to do much with 
other students-who are "loners" 

13. The usual social atmosphere or feeling in the classroom 

1 4. The respect teachers and students have for one another 

15. Teachers who don't seem to care about the personal and 
educational problems of their students 

16. Teachers who put too much pressure on their students to get 
good grades 

1 7. Unfair treatment of students by teachers 

18. Teachers who won't admit making mistakes or think there is 
only one right answer lo every question 

1 9. Teachers who complain about other teachers 

20. Teachers who seem bored with teaching 
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IS THIS A PROBLEM IN YOUR SCHOOL^ 
(Put an X for each problem in the box 
under your choice.) 






0 
£ 




A 


c 
3 


b 


1 


2 


3 


4 


5 


6 


1 


2 


3 


4 


5 


6 


1 


2 


3 


4 


5 


6 


1 


2 


3 


4 


5 


6 


1 


2 


3 


4 


5 


6 


1 


2 


3 


4 


5 


6 


1 


2 


3 


4 


5 


6 


1 


2 


3 


4 


5 


6 



2 1 . Teachers who are usually boring 

22. The way the principal gets along with the students 

23. A feeling in the school that conformity and orderliness among 
the students are more Important than freedom and mdividuality 

24. Rules for students that are not clear but are vague and indefinite 

25. The amount of influence student opinion has on the way the school 
is run 

26. The way students are assigned to classes, graded and promoted 

27. Unfair treatment of students by the principal or by the people in 
his office 

28. The way the teachers get along with parents 

29. The way the people in this neighborhood feel about the school 

30. Cheating 

31. Cutting class 

32. Absenteeism 

33. Dropouts 

34. Vandalism 

35. Theft 

36. Drugs 

37. Alcohol 

38. Delinquency 

39. Profanity 

40. Violence or threats of violence 
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43. Discipline 

44. Dress code 

45. Useless courses 

46. Not enough counseling .... 

47. Not enough medical services 



50. Other 



1 


2 


3 


4 


5 




6 


1 


2 


3 


4 


5 




6 


1 


2 


3 


4 


5 




6 


1 


2 


3 


4 


5 




6 


1 


2 


3 


4 


5 




6 


1 


2 


3 


4 


5 




6 


1 


2 


3 


4 


5 




6 


1 


2 


3 


4 


5 




6 


1 


2 


3 


4 


5 




6 


1 


2 


3 


4 


5 




6 


1 


2 


3 


4 


5 




6 



41. Ethnic tensions 


1 


2 


3 


4 


5 


6 




1 


2 


3 


4 


5 


6 



1 


2 


3 


4 


5 


6 


*1 


2 


3 


4 


5 


6 


1 


2 


3 


4 


5 


6 


1 


2 


3 


4 


5 


6 


1 


2 


3 


4 


5 


6 





1 


2 


3 


4 


5 


6 


49. Class size 


1 


2 


3 


4 


5 


6 
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Procedures for Surveying School Problems 



Table 1 

Common and Unique Items on Staff and Student Questionnaires 

NOTE: A dash (-) indicates items with no counterpart in the other quesiionnaire; an asterisk (*) 
indicates items with minor modification t.i wording. Items are grouped into 12 potential 
problem areas. The first six obtain estimates by staff and students of the problems they 
see in the characteristics of, and the interrelationships among, students, staff, principal, 
and community. The last six obtain the staff and ttuot>nts' rating, as to severity of a 
variety of problems associated with students, teachers, administrator, facility, 
and community. 



School Attractiveness (SA) 

Staff -, 

Student 1, 2, 3, 4, 5, 

Student Characteristics and Relationships (SS) 
Staff 1, 2, 3, 4, 5, 

Student 7, 8, 9, 10, 

Teacher-Student Relationships (TS) 
Staff 7, 8, 9, 10, 

Student 13, 14, 15, 16, 

Teacher Chatacteristics and Relationships (TT) 



11, 

11, 
17, 



6 

12 

12 
18 



Staff 
Student 



63, 
45*, 



17, 18, - 



21 



22, 



40, 41, 
32, 33* 



23, 
25, 



29, 30 



35, 36 



13, 14, 15, 16, 
19, -, -. 20, 

Principal-Student Relationships (PS) 
Staff 19, 20, 21 
Student 22, 23, 24, 

Principal-Teacher Relationships (PT) 
Staff 25, 26, 27, 28, 
Student -, -, -, 

School-Community Relationships (SO 
Staff 31, 32, 33, 34, 
Student 28, 29, 

Student Problems (SP) 

Staff 37, 38, 39, 
Student -, 30, 31, 

Community Problems (CP) 

Staf^ 51, 52, 53, 54, 55, 56 
Student -, 41, -, 

Administrator Problems (AP) 
Staff 60, 61, 62, 
Student 43, 44, 

Facility Problems (FP) 
Staff 67, 68 
Student 48, 49 

Teacher Problems (TP) 
Staff 69, 70 
Student 



24, - 
26, 27, 



42, 43, 
34, 35, 



44, 45, 
36, 37, 



46, 47, 48, 49, 50 
-, 38, 39, 40 



57, 58, 59 
42, - 



64, 65, 66 
-, 46*, 47* 
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The Instruments 



Introducing the Instruments Into a 
School and Obtaining Data 

Several preliminary steps' are recommended to introduce the School 
Demographic Information Form and the two questionnaires into a schoo'. A 
description of what should occur and an occasional explanation for the 
activity follow. 

(1) The consultant, personally or through a supervisor, should receive the 
approval of the superintendent or a designated representative to work in 
the school. 

(2) The consultant should contact the principal of the school, offering his or 
her services as a mental health consultant and arranging for an appointment. 

(3) In the first meeting with the principal, the consultant should describe 
the services he and his facility can offer and the manner in which plans for 
consultation can be made. 

(4) If the principal agrees, at least tentatively, to accept the services of the 
consultant, copies of the School Demographic Information Form and of the two 
questionnaires should be left with him. (The demographic form is given to the 
principal prior to the actual interview so that he can familiarize himself with the 
various topics to be discussed and can decide whether he would like other staff 
members to participate in the interview. The questionnaires are left with him so 
that he can review their content.) The consultant should assure the principal that 
a summary of all questionnaire data will be provided, so that they can review the 
data together. 

Arrangements should also be made for a second meeting in which the 
Demographic Information Form will be completed. The principal may also be 
interested in filling out one of the staff questionnaires. (If the principal completes 
a staff questionnaire, he and the consultant will be able to compare the principal's 
perceptions with those of the staff.) 

(5) Before the second interview with the principal, the consultant should fill 
out as much of the interview form as he can from his own knowledge or from 
available sources, within his agency. He can verify this information during his 
interview with the principal. 

(6) During the second interview, both the principal and the consultant 
should have a copy of the School Demographic Information Form. When the form 
has been completed, arrangements should be made for administering the 
questionnaires to the teacher? and students. The principal, alone or with the staff, 
should decide whether to use time at a staff meeting to complete the staff 
questionnaire, or ask the teachers to complete it on their own time. (It is 
preferable to administer the questionnaire to the teachers as a group so that 
(a) any questions they may have can be dealt with immediately and directly, (b) a 
maximum number of the questionnaires will be returned, and (c) the teachers are 
aware of the principal's interest in the questionnaire. If the questionnaire is given 
as the final item in a staff meeting, the teachers can leave as they finish.) 

' Again. Beisser and Green, op.c/f., describe these preliminary steps in considerable detail 
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Procedures for Surveying School Problems 



Two decisions have to be made about the student questionnaire at this 

point: 

(a) Whether the principal wants to see the summary of the data 
from the teachers' questionnaire before making plans for giving 
the student questionnaire. 

(b) The extent to which the principal wants to involve the staff in 
planning for the administration of the student questionnaire. 

(7) The principal and the consultant jointly should discuss the staff 
questionnaire with the teachers before it is administered. Preliminary remarks and 
discussion should: 

y (a) Explain the reason for the consultant's presence. 

(b) Sketch out the kinds of mental health activities that are available to 
the school. 

(c) Emphasize the need for systematic planning of mental health 
interventions. 

(d) Outline the procedures for assessing the school's needs and the 
specific purposes of the staff questionnaire. 

(e) Mention the plan to administer a similar questionnaire to students. 

(f) Assure the anonymity of respondents. (An envelope should be 
provided with each questionnaire in which the completed form can be sealed to 
reassure the teachers about the confidentiality of their responses.) 

(g) Explain that both teacher and student data will be summarized and 
made available to the teaching staff for review. 

(h) If the principal has already filled out the staff questionnaire, 
mention this fact. 

(8) When the questionnaires are completed, the consultant or some member 
of the mental health facility can analyze the data, following the procedures 
outlined briefly in the next section of this manual and given in full detail, with an 
example, in Appendix A, or by using the computer program, which is presented in 
Appendix B.' 

(9) Depending upon decisions reached in Step (6), either of two procedures 
may be followed: 

(a) The consultant will meet with the principal to discuss the data from 
the teacher questionnaire; then the student questionnaire will be administered, the 
data analyzed, and the results reviewed in a second meeting with the principal. 

(b) Alternatively, the student questionnaire will be administered and 
the data analyzed; then the consultant will meet with the principal to review the 
data from both questionnaires at the same time. 

Option (b) is preferable. It not only uses the time of the principal and 
the consultant more efficiently, but also introduces the teacher and student 
responses for simultaneous consideration and precludes any crystallization of a 
point of view based on teacher responses alone. In addition, the similarities and 
discrepancies in the perceptions of teachers and students become immediately 



'The programs, written in FORTRAN IV, can be used at any facility where a FORTRAN 
compSer is available. The programs for staff and student questionnaires contain 103 and 102 
FORTRAN statements, respectively, and each operating program fits easily into 8288 bytes. 
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apparent. Discrepancies are particularly important because they may indicate a 
need for more information, and may shape the approach to be taken 
in intervention. 

(10) The student questionnaire can be administered to a random sample of 
approximately 50 students. This number is quite adequate without being too 
cumbersome to analyze. While the questionnaire could be administered to the 
entire student body, processing the data would be quite time-consuming where 
keypunch and computer facilities are not available. 

The sample should come from students in the highest grade level in the 
school; they are probably the best informed and have a more mature perspective 
about the school. The sample should not consist of students in a single classroom 
unless there is strong reason to believe they are representative of the students at 
that grade level (health or physical education classes, for example, are often made 
up on an essentially random basis). Under no circumstances should the sample b€ 
selected on a basis such as teachers' nominations. If a sample selected at randon* 
is not possible and there is no clear alternative to selection by class, the sampk 
should consist of two or three smaller classes that will represent the whole range 
of students at that grade level. 

(11) Both the staff and the student data should be reviewed with the teaching 
staff to identify areas that are perceived as problems by teachers as a group and 
by students as a group. When the problem areas have been identified, discussion 
can begin on priorities for programs of intervention. 



Analyzinj the Data and Settiii§ Priorities 



Analysis of QuesfioniMire Oafa 



The statistical procedures to be followed in analyzing the data are outlined 
below. (As noted earlier. Appendix A contains a detailed description of these 
statistical procedures, and Appendix B contains keypunched instructions and a 
computer program.) The product of this analysis is a list of problems ordered 
by salience. 

The steps to be taken in the analysis of either questionnaire are: 

(1) Tabulate the responses to each item. 

(2) Find the mean rating for each item. 

(3) Find the overall mean rating and the overall variance of the 
responses (for the questionnaire as a whole). 

(4) Standardize the mean item scores by transforming them into 
T*scores with a mean of 50 and standard deviation Ox 10. 

(5) Find a mean T-score for each problem area (see Table 1 for 
identification of areas). Figure 1 shows a summary sheet to be used 
^rsr rpcording T-scores and their means. 
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School Problem Area Surveys: 
Summary of Staff and Student Responses 
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(6) Order the sections (areas) from high to low on the basis of their 
mean T-scores. 

(7) Order the items from high to low on the basis of their T-scores. 

(8) If desirable, draw a school profile of problem areas. Figure 2 shows 
a form to be used for this purpose. 

(9) If desirable, T-scores of both staff and student responses can be 
recorded on a blank questionnaire in association with the items 
they represent. 

As mentioned earlier, each of these steps except (9) is shown in detail in 
Appendix A, using data obtained with the staff questionnaire. 
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Sittins Priorities for Mental Healtli intervention 



Two steps are involved in identifying those problems for which mental health 
intervention would be potentially most relevant: (a) identifying the most salient 
problems, and (b) establishing priorities for the most salient problems. The 
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salience of a problem is inferred from the data gathered using the three 
instrumentsHJie staff questionnaire, the student questionnaire, and the 
Dem(^phic Information Form. The priority assigned to a problem depends upon 
the extent to which it disrupts, or may disrupt, achievement of the educational 
goals of the school.' 

Salience . The relative salience of each problem is inferred from its ratings by 
the student and staff respondents and from the principal's ordering of problems 
by importance in his summary statement in the Demographic Information Form. 

The more often the respondents rate an item toward the "extreme 
problem" end of the scale, the lower its mean rating will be. Items with means 
that are low relative to the means of the other items in the questionnaires are 
assumed to have greater salience for the respondents. 

Priority . The priority of a problem depends upon its salience and the extent 
to which it disrupts, or has the potential for disrupting, attainment of the 
educational goals of the school. The extent to which a problem may be disruptive 
depends, in turn, upon ihe nature of the problem and the number of persons 
affected by it. To assign priorities, it is recommended that the five to ten most 
salient problems from all three sources of data be rank ordered with respect to 
their actual or potential disruptiveness. 

Problems which would be ranked highest are those which have the 
potential to render the school inoperable. Racial conflict and student unrest are, 
under some conditions, such problems. 

Generally, next in order of disruptiveness are those problems which 
preclude effective learning by sizable numbers of students. Linguistic barriers 
between students and teachers (in the form of a foreign language or even of 
non-standaid English) and drug abuse are sometimes such problems. Since they 
virtually preclude communication between teachers and students, they interfere 
with learning. 

Next, problems that might be judged moderately disruptive are systemic 
problems of the school which interfere with the successful accomplishment of its 
mission. Difficulties in interpersonal communication among the school staff, or of 
role definition, inter-group conflicts over school goals and policies, inequities or 
uncertainties in policy setting or in decision making are examples. 

Finally, rankings at the lower end of the disruptiveness scale might be 
assigned to problems that interfere with the learning and/or socialization of 
individual students— for example, the underachieving student, and the 
habitual absentee.^ 

Ideally, the principal, representatives of the school staff, and the 
consultant will meet jointly to establish priorities, that is, to rank order the most 
salient problems with regard to their disruptive effects. A convenient way to 
display the most salient problems to be reviewed in this meeting is to list, in 

' The goab of the school may be interpreted in the broadest sense and may vary from school 
to school. Some schoob may focus their goals strictly on academic achievement; others may 
emphasize the development of self-concepts, career orientation, etc. 

These four categories of problems are arranged as a hierarchy based upon the premise that 
the most relevant activity of the mental health consultant and the school stoff is directed at 
reaching the largest **audience" or client group. 
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order, the five to ten items with the highest salience (lowest mean rating) from 
the two questionnaires and the last page of the Demographic Information Form 
(the principal's Summarization). The data from the three sources (the staff, 
students, and principal) should be given equal weight in rank ordering the priority 
of the various problems. 

Setting priorities for intervention in accordance with the estimated 
disruptive effects of probicmF is essential as a first step in planning relevant 
indirect services. However, once the priorities have been established, the final 
stage in planning a program of intervention remains, although it is not within the 
scope of this manual. Appropriate treatments for each of the high-priority 
problems must be considered and a final selection made based upon: 

• The cust in time, personnel, and mon»^y for each intervention. 

• The feasibility and acceptability of each intervention to the 
school and the community. 

• The pay-off or the anticipated outcome of each intervention. 

While guidelines have been presented here for setting priorities, the final 
judgment on what problems will be addressed and what strategies will be used to 
deal with them must rest with the principal and his staff and their personal 
knowledge of the school. The procedures and instruments provided in this manual 
ensure only that a very broad inspection will be made of the potential problems 
of the school. The likelihood that a program of indirect services will be launched 
which deals with trivial problems, or affects only a very small number of persons, 
or reflects no more than the personal biases of one or two people will be reducec 
if these procedures rre followed. 

The ni^chodology described up to this point has been limited to 
determining »iie direction of intervention within a school. Similar procedures 
would be employed when the methodology is used to assess problems of several 
schools in order to allocate resources among them. The superintendent's office 
and/or a mental health facility would inspect the highest priority problems of 
several schools and select a school (or schools) for intervention by ordering the 
problems preserited in f^mis of their relative disruptiveness. The same constraints 
of cost, feasibility, and pay-off would condition the final decision on allocation 
of resources 
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THE RaiABiiiry m 

VAUPITV OF MEASURES 



In this part of the manual, the reliability and the validity of the question- 
naires are discussed. The general meanings of reliability and validity as they are 
applicable to these instruments, are explored. Findings on reliability and validity 
are presented. 

For the reader interested in the more technical details on which conclu- 
sions are based. Notes have been included in a separate section at the end of 
the text. 

ReliabilHy 

For the most part, the reliability of an instrument is concerned with the 
question: "Can I believe what the scores tell me?" There are two ways this ques- 
tion can be asked, and each form of the question deals with a different factor 
of reliability: 

• The first form deals with internal consistency measures 
of reliability: "Are the scores telling me anything?" 

• The second form deals with some form of test-retest 
reliability: "Can I rely over time on what the scores 
tell me?" 



MeasurM of Reiiabiiily 



1. "Are the scores telling me anything? " The great variety of internal 
consistency measures of reliability show, by one formula or another, that the 
distribution of scores obtained with an instrument is not likely to be due to 
chance, that the responses made to the items are patterned, that certain responses 
are given more often than chance would allow to some items than to others. 
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While it may seem trivial to demonstrate that the response matrix is not a random 
aggregation, it is necessary to show that there is some consistency in 
the responses.' 

2. "Can I rely over time on what the scores tell me?" Conventional 
psychometry is directed at measuring relatively stable characteristics of persons; 
therefore, this question is usually answered with some form of test-retest 
reliability. Since the problems in a school can be expected to shift over time, 
sometimes very rapidly, test-retest reliability has limited utility for these 
instruments. If you questioned one random half of the respondents of a school 
one week and the other half a week later, a high correlation between them would 
indeed indicate good reliability over time, but a low correlation could suggest 
only that something had changed in the meantime, rather than that the 
instruments display poor reliability over time. 



Reliabilify AMiysis 



Measures of internal consistency must be confined to data collected within a 
school and the resultant statistics are specific to that school. It would be illogical 
to calculate measures of internal consistency using the pooled data of several 
schools, since internal consistency relies upon the extent of agreement about 
conditions in a particular school. 

To get some notion of the reliabilities we might expect from our 
questionnaire, we analyzed thj data from the staff and the ninth grade students 
of one school selected at random from among the 13 schools used for the final 
testing of the questionnaires. The questionnaires administered in these schools 
differed in only minor particulars from the final printed versions contained in this 
manual. From this school we had 37 complete staff questionnaires and 39 ninth 
grade student questionnaires. 

Reliabilities obtained were .95 for the staff questionnaire and .82 for the 
student questionnaire. (Note #1 presents the results of this analysis in 
more detail ) 

Based upon this and other analyses we have done, we can state the 
following conclusions: 

(1) Reliabilities of a high order can be obtained with the 
questionnaires. 



'No n^mple statistical test will tell whether scores that are reliable constitute a 
measurement, however. Both the stimuli (items) and the response options must he examined to 
see whether they make sense as a measure. Our questionnaire items are all concerned with 
potential problems in schools. The respondents indicate how much of a problem they think 
each one is in their school. In the data collected thus far in some 50 schools, the respondents 
in all cases indicate by their choices that they agree some itemp rra more of a problem 
than others. 
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(2) Reliabilities will vary from school to school depending on whethe; 
or not the schooi has some outstanding strengths and weaknesses.' 

(a) In a school with no problems, reliability measures are likely to 
be low. 

(b) In a school with severe problems, reliability measures are likely 
to be high. 

(3) Students will tend to give less reliable data than staff, although in a 
school with outstanding and severe problems, student consensus will 
be bette]M;he reliability higher-and they will show greater 
agreement with the staff evaluation of items.^ 

Although we collected no test-retest data, it is reasonable to consider how 
much random fluctuation might be expected in a single item mean from one time 
to another. Based on the analysis in this one school, a shift in an item's T-score 
of more than about 5 points in the statf questionnaire, or 10 in the student 
questionnaire would be a fairly rare chance event, unless the number of 
respondents is quite small (15 or less). Cl^anges lai^er than those may indicate real 
changes in the respondents' evaluation of ti:e problem. These estimates, however, 
are subject to revision with further experience. 



Valldiiy 

Our measures appear to tell us something about the current state of affairs in 
a school with reasonable reliability. Is what they tell us true? Are the measures 
valid? While the validity question is usually asked in terms of truth, it is more 
sensible to ask it in terms of utUity for some purpose. Either way implies that the 
data can be checked or validated against some external criterion: the first, against 
some other, presumably less fallible source of information; the second, against 
some ultimate measure of utility. Each of these concepts is taken up on the fol- 
lowing pages. 



'in a school with few or no outstanding strengths or weaknesses, lower internal 
consistency will be obtained, for the reason that there can be little agreement among 
respondents if there is little that is noticeable enough for them to agree upon. A measure can't 
tell you anything if there is nothing to tell. In conventional psychometrics, this difficulty 
corresponds to the very common problem of restricted range. The instruments appear to 
provide reliable data if there is something to discriminate and if the number of respondents is 
reasonably large. 

^In these data, using the 40 items common to the two questionnaires, the Staff^tudent 
correlation was only .26. However, in an analysis of 32 schools studied earlier, ihe general level 
of response (overall mean) on staff and student questionnaires was found to conelate .62. 
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eross-Validaiioii 



The staff and student questionnaires might be checked against one another as 
independent sources or against the principal's statements in his or her interview. 
However, the staff, the students, and the principal may have points of view that 
are so different as to vitiate any comparisons. Teachers seem to complain about 
class size fairly often, while students never do. Teachers are concerned with 
tactical classroom problems and principals with strat^ic school problems, and 
while each may appreciate the other's problems, they seldom take them on as 
their own. 

Our data do surest that Teacher-Student correlations, based on the T-scores 
of the common items, may be higher in more troubled schools. That suggests in 
turn that cross-validation internal to the school may depend upon the presence of 
obvious and severe problems which have a considerable and identifiable impact 
upon everyone in the school. In a school with no severe problems, each of tho 
three respondent groups may be preoccupied with its own problems and, as a 
consequence, show little cross-validity. In principle, at least, the only source of 
information about what teachers, students, and principals believe are the problems 
of a school are the teachers, students, and principals. While the groups may differ 
considerably, the perceptions or beliefs of one group cannot be used to invalidate 
the percepti: ns or beliefs of another. 

Face validity should not be overlooked. These questionnaires direct 
respondents to express their feelings about a series of potential problems, and, 
unless the respondents are deliberately trying to deceive the investigator, it must 
be assumed that they have expressed their feelings. Therefore, the instruments 
have face validity (barring deception) as measures of the respondents' feelings 
about the problems listed. This does not guarantee that the feelings expressed are 
consonant With reality, however. A school where the "ethnic tensions" item gets a 
low (unfavorable) score may be having a problem of ethnic tension or the 
respondents may have given ethnic tensions low ratings because they were overly 
sensitive to the general issue or overly reactive to minimal cues. In any case the 
immediate question to be dealt with is the respondents' feelings. Ultimately, 
whether feelings or ethnic tension or both are to be treated will depend upon the 
particular situation in the school. 



UHRfy 

It is better to ask whether the questionnaire data suggest useful points of 
attack in planning programs of intervention. Do the questionnaires fulfill their 
purpose as instruments for a mental health consultant or other persons to use in 
order to give the school staff a systematic and comprdiensive view of what they 
themselves (and the students) see as the problems of the school and to stimulate 
cooperative action by the staff directed at dealing with those problems? 
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Although the instruments have had, as yet, only limited use as a preliminary 
to planning programs of intervention, what use they have had has been reported 
to be quite successful. 



Evidence About the Utility of the Questionnaires 

We carried out several analyses on groups of schools to determine, for the 
first six areas of the questionnaires, whether schools tend to show similar profiles, 
whether some areas tend to be seen as problem areas in all schools sampiad, and 
whether the schools are differentiated by the various areas of the questionnaires. 
(Note #2 presents the results of these analyses in more detail.) 

From these analyses we can state the following conclusions: 

(1) School profiles are mildly similar for staffs and quite similar for 
students. (In a sample of 33 schools, the staff **profiles" correlated 
.34 and the student profiles correlated .64. In a sample of 10 
schools, using the more nearly ^inal forms of the questionnaires, the 
correlations were .56 and .65 for stafi and students, respectively.) 

(2) Some areas tend to get consistently unfavorable ratings (Student- 
Student for the Staff and Principal-Student for the Students) and 
others consistently favorable ratings (School-Community for Staff 
and Students) across all of the schools in the samples. 

(3) The questionnaires do discriminate differentially among schools in 
the various areas. 

These findings support the presumption . of validity, not only of the 
questionnaire as a whole, but of the sections directed at each potential problem area. 



Summation 

These questionnaires appear to be both a reliable and a valid means of identi- 
fying major sources of discontent in a school, provided the respondents are willing 
to cooperate in answering the questions, as they usually seem to be. 
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Note ^1 —Sample Analysis of Data 



For one school, analyses of variance were computed on the responses to each of the first six sections 
of the two questionnaires and to the two questionnaires as wholes. We analyzed the responses into three 
orthogonal components: Items^ Respondents^ and Items by Respondents Interaction which served as the 
Error term. 

The Error components in the first six sections of the two questionnaires were: 



Section 

Schooi Attractiveness 

Student-Student 

Teacher-Student 

Teacher-Teacher 

Principal-Student 

Principal-Teacher 

School-Community 



Staff 



.41 
.68 
.55 
.94 

.75 
.77 



Student 

2.01 
1.28 
1.38 
1.09 
1.78 

1.15 



The Items components were: 



Section 

School Attractiveness 

Student-Student 

Teacher-Student 

Teacher-Teacher 

Principal-Student 

Principat-Teacher 

School -Community 



Staff 



14.86 
12.04 
{ 5.25) 
(16.48) 
3.50 
( 6.91) 



Student 

4.90 
4.81 
4.32 
(45.23) 
( 9.37) 

(13.12) 



(Values in parentheses are not comparable because both the item content and the number of items varied 
between the two Questionnaires.) 

The reliabilities (interna! consistency, computed by analysis of variance) of the various sections 
and of the entire questionnaires were: 



Section (or Area) 


Staff 


Student 


School Attractiveness 




.59 


Student-Student 


.97 


,73 


Teacher-Student 


.94 


.68 


Teacher-Teacher 


.89 


.98 


Principal-Student 


.94 


.81 


Principal-Teacher 


.79 




School-Community 


.89 




Total Questionnaire 


.95 


.82 



"Total Questionnaire/* of course, includes several sets of brief items as well as the six sections listed. 

'Winer, B.J., Statistical Principles in Experimental Design, McGraw-Hill Book Company, 1962. 

^In the 37 staff questionnaires, 161 of the responses (6.2%) were missing; in the 39 student 
questionnaires, 36 responses (1.9%) were missing. To simplify the computations we supplied scores for 
the missing responses, using the two response values on either side of the mean of the item in question. 
The number of de^ees of freedom in the various analyses was re(*uced accordingly. 
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In the section of the staff questionnaire with the lowest reliability (Principal-Teadier, .79) the 
T-scores of the individual items were 118, 68, 108, 97, 89, and 103, showing that the staff was 
uniformly well pleased with its relationships with the principal. Obviously, since reliability is a function 
of both consensus and item differentiation, such scores can yield only a relatively low reliability. 

As would be expected from their greater lack of consensus (the Error components), the student 
reliabilities are uniformly lower than the staff except in two areas. However, in the one, 
Teacher-Teacher, the student questionnaire contains only three items, two of which were rated very high 
and one very low; in the other, School-Community, there are only two items, one rated very high and 
the other about average. Obviously, when there are very few items, reliability is only a moderately 
useful concept. 

Note #2-'Results of Sample Comparison of Schools 

In a pilot study, data were obtained from the staffs and students of 33 junior high schools 
responding to earlier and longer versions of the two questionnaires, one student questionnaire, and three 
forms of a staff questionnaire. For this analysis, the items in the questionnaires were classified into the 
six areas or categories used in the final versions of the questionnaires (Student-Student, Teacher-Student, 
and so on). The T-scores of the Uems in each area were then averaged for the staff and students of each 
school. This process gave six mean T-scores for the staff of each school and four for the students, who 
had no Teacher-Teacher or Principal -Teacher items. Because there were three different forms of the staff 
questionnaire, the number of items entering into each mean T-score varied rather considerably. 
Accordingly, the mean T-scores were all restandardized on the basis of the number of items entering 
into each. This made the means more nearly comparable. Analyses of these two sets of mean T-scores 
(using the Schools by Areas Interaction for the Error term, as is appropriate for a random effects 
design) gave the following results: 



Source 


df 


MS 


F 




Schools 


32 


150 


< 1 


NS 


Areas 


5 


12845 


17.94 


<.01 


SxA 


160 


716 






Total 


!97 








Within Schools 


165 


1084 


r 
.34 





Source df^ MS^ f_ _P 

Student: Schools 32 74 < 1 NS 

Areas 3 52074 59.86 < .01 

SxA 96 870 

Total 131 

r 

Within Schools 99 2421 .64 



As expected, the Schools effect is not significant since overall differences between schools are eliminated 
by the initial standardization. The highly significant Areas components indicate that some areas tend to 
get consistently unfavorable ratings (Student-Student for the staff and Prin< ' )aisStudent for the 
students) and others consistently favorable ratings (School -Community ior staff and students) across all 
of the schools in the sample. The intraclass correlations (r) show the extent of this trend more clearly. 
The average intercorrelations among schools of .34 for staff and .64 for students show that school 
profiles are mildly similar for staffs and quite similar for students. 

SI 
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In a later analysis of data collected from the staffs of ten schools and the students of six schools 
(using the newer versions of the questionnaires, which closelv resemble the final versions), the following 
results were obtained :< 



Source df MS P ^ 

Staff Schools 9 159 <1 NS 

Areas 5 2508 13.63 < .01 

SxA 45 184 

Total 59 

r 

Within Schools 50 417 .56 



Source df MS F P 

Student. Schools 5 108 <1 NS 

Areas 4 2318 12.40 < .01 

SxA 20 187 

Total 29 

r 

Within Schools 24 542 .65 



These analyses are strikingly like the first and yield the same conclusions. 

As will be noted, the mean squares obtained in the second pair of analyses are all markedly 
smaller than those obtamed in the first. The mean T scores used in the second analyses were based on 
six items each and did not require restandardization to mak e them comparable to one another. Their 
standard deviation was, therefore, not 10 but 4 (i.e., \jl00 -J-6). 

In this type of analysis, the interaction mean square can be expected to be equal to or something 
lesc than the square of the standard deviation of the original observations, which is 100 for the first pair 
of analyses and 16.7 for the second. In all of these analyses, however, the SxA interaction terms are 7 
to 10 times larger than expectation. A large and real interaction between Schools and Areas is apnarent 
in these data. Despite the general similarity of school profiles, there are large and highly significant 
differences among schools within the areas; the questionnaires are, in fact, discriminating differentially 
among schools in the various areas. 
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Appendix A 



DIRECTIONS FOR HAND TABULATION AND 
ANALYSIS OF QUESTIONNAIRE DATA 



The first time you receive a bundle of completed questionnaires, you may be 
inclined to tabulate the responses directly onto a blank questionnaire form. However, 
unless the bundle is very small, suppress that impulse. The number of errors you can 
make in such direct tabulation is sizable, and tracking down responses missing from the 
tabulation is very time-consuming. 

The following steps are recommended for data analysis when it must be done 
by hand: 

1. Transcribe the responses on each questionnaire to a work sheet which can display 
all of the responses of all of the respondents in compact, accessible, and usable form. 

A sample page of raw data from one completed staff questionnaire is shown in 
Figure A-1. Data in Figure A-1 have been recorded in Column A of the work sheet shown in 
Table A-1. (Columns B-J show the responses of nine additional staff members.) To trans* 
cribe the data from a questionnaire to the work sheet, record in the appropriate row for the 
item and the appropriate column for the respondent, the number in the box that has been 
marked on the questionnaire. If no response is made to an item, record a dash If 
Columns (Exceptionally Good) is checked, record it as a 5 with a 6 in parentheses 
following, as in 17-G in Table A-1. 

In transcribing the raw data, discard questionnaires with an excessive number of 
blanks, many multiple responses, or other evidence of carelessness or inappropriate 
response sets. (Such questionnaires have, thus &r, been uncommon.) There is no need to 
put identification numbers on individual questionnaires unless you think you may want 
to go back to check the accuracy of your transcription. Normally, once you have 
transcribed the raw data, you can dispose of the questionnaires, unless they contain 
written comments you want to save. 

2. In Table A-1, count the number of times each response was made to each item; 
then record this information on a surntf^ry sheet, by item, under the appropriate 
Response Value column (see Table A'2.) 

The response frequencies of the items in Table A-1 have been recorded in 
Table A-2. For example. Table A-1 shows that, for Iteml, five respondents checked 
Response 2 (Considerable Problem) and live respondents checked Responses (Moderate 
Problem). Therefore, in Table A-2, in the row for Item 1 there is a 5 under Response 
Column 2 and a 5 under Response Column 3. 

3. Find the mean rating or average response made to each item. 

You will need to make two summations to do this: 

• The first summation, shown in Table A-2 as Sf, is the number of 
respondents who gave scorable responses to the item. It is found simply 
by adding the numbers in each row, except those in the No Response 
column. For example, in Table A-2 one of the 10 respondents did not 
answer Item 7; therefore, for Item 7, Sf = 9. 
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Sample Page of Raw Data 



IS THIS A PROBLEM IN YOUR SCHOOL? 
(Pui an X in the 'appropriate column.) 




1 . The way the students yet along with one another 

2. The number of students wh*- f like going to school and don't 
do their school work 

3. Students from ethnic minorities 

4. Friction or hostility between group:^ of students 

5. Capable students who feeM^ { ^ j school is pretty much a 
waste of time 

6. The number of students who don't seem to do much with ot*" r 
students-who are "loners" 

7. The usual social atmosphere or feeling in the classroom 

8. The respect teachers and students have for one another 

9. Teachers who don't seem to care about the personal and 
educational problems of their students 

10. Teachers who put too much pressure on their students to get good 
grades 

1 1 . Unfair treatment of students by teachers 

12. Teachers who won't admit making mistakes or think there is only one 
right answer to every question 

1 3. Teachers who complain about other teachers 

14. Disagreements among the staff on the proper educational goals 
for the school 

15. Disagreements among the staff on the proper balance between 
traditional and innovative approaches to teaching 

16. Communication among the school staff 

17. Teachers who seem bored with teaching 

18. Older teachers who are reluctant to accept newer teachers 
as colleagues 





2 


V/ 


4 


5 


6 




t * 


3 


4 


5 


6 


1 


X 


3 


4 


5 


6 




2 




4 


b 


6 






3 


4 


5 


6 




2 


3 




5 


6 






3 


4 


5 


6 






3 


4 


5 


6 






3 


4 


5 


6 


1 


2 


3 


4 


y 


6 




2 


3 




5 


6 




2 


3 


4 


K 


6 




2 


3 


X 


5 


6 




K 


3 


4 


5 


6 




K 


3 


4 


5 


6 


2 




4 


5 






/ 


3 


4 


5 


6 


1 


2 


3 


X 


5 


6 



er|c 5" 
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Table A-1 
Sample Work Sheet 





Respondents AO {Staff or Students) 


Item 


A 


B 


c 


0 


E 


F 


G 


H 


1 


J 


1 


3 


;l 


3 




3 






3 




3 


2 


a 




3 


\ 


0. 










1^ 


3 






'J 


5 


3 


J. 








3 


4 


0 






3 


3 






3 






5 


X 




\ 


1 




a 


3 


a 




1 


6 


a 


3 




3 








4 


3 


3 


7 






3 


3 






3 


3 


3 




8 


Q 


/ 








•1 






1 




9 




X 










5 


^ 


^ 




10 




3 




5 






3 


5 






11 




w 




5 


5 


3 










12 


5 








5 


3 




;j 






13 










5 


H 








s 


14 










5' 












15 










r 






3 




a 


16 




^. 


3 




5 






*:.> 






17 








5 




-3 










18 








S 


5 


5 






V 
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• The second vammation is shown as S fx. It is the sum of all the 
response values recorded for each item. It is obtained for each item by 
multiplying the frequency of ^^ach response by its value and summing 
these products. For example, for Item 1 in Table A-2: 

0 (the frequency) times (the value) = 0 

5 X "2'* = 10 

5 X "3*' = 15 

0 X "4'* = 0 

0 X "5" = 0 

0 X "5''(6) = 0 

Sf = 10 ^fx = 25 

NOTE: Response 6 (Exceptionally Good), coded 
as 6(6), has the same value as Response 5 (No 
Problem at All). 
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Table A.2 

Sample Summary Sheet and Computations 



Responso Value 



Item 


1 


2 


3 


4 


5 


5(6) 


No 
Response 




No. of 
Responses 


Total 

1 • 1 ISO 

Value 


Rating 


T'Score 


1 


















to 








o 


/ 


f 


/ 


— 










/o 


^0 






Q 

o 




3 


4 




I 








/o 








4 








— 








/o 








5 


5 




X 


— 










/c 


n 






6 






7 












/o 






sf 


7 




9. 


7 




MM. 








9 






34> 


8 




J 














/o 








9 






/ 




/ 








/c 






r 


10 




















46 




7/ 


1 1 














/ 




9 


37 






12 














1 






37 




7y 


13 




/ 




7 












4^ 




7/ 


14 


— 


/ 






/ 




t 






33 


3C7 




15 




3 


/ 




/ 










a-C 


3'U 




16 




^ 










1 




9 


SI 


344 




17 




1 








/ 








3f 


596 




18 


















/c 


4^\4'SC 




Ho. of 
Responses 
(2f) 

Totals 

Sums of 
Squared 
Values 
|2fx2) 




4! 

u4 




7J6 




/ 

AS 


7 


S2f 22fx X T 

f7i i'^y 3'M SO 



Summations and Scores 



a = Z2f - 173 
o - SSfx - SCif 
c - 2Sfx2. 3,o^X 



a2d 



^k>. Items 
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The mean rating of an item, shown as x is its 2 fx divided by its Sf. In 
25 

the example = 2.5, which is the mean for Item 1. 

All of these values, Sf, Sfx, and x are shown for each item in Table A-2. 
4. Determine which item means are high enough to suggest a ^'really'' favorable 
situation or low enough to point to a '"really"' unfavorable situation. 

It is obvious that Item 5 in Table A-2, with a mean of 1.9, is the "worst," and 
Item 18, with a ixiean of 4.5, is the **best," with the other items somewhere between. 

How high or how low must an item mean be to be considered **really** 
favorable or **really** unfavorable? 

Consider a hypothetical experiment in which you could give the same question- 
naire over and over again to the same teachers without their remembering you had done 
so. Assume that conditions in the school remain substantially the same during the period 
of the experiment. On the questionnaire as a whole you would expect to get about the 
samo distribution of responses-the same number of responses to Iteml, the same 
number of responses to Item 2, the same number to Item 3, and 4 and 5— each time the 
questionnaire was administered. However, you would not be siurpri^ if the distribution 
of responses to a given item (hence the item mean) varied somewhjat from one adminis- 
tration of the questionnaire to another. 

The amount that an item mean can be expected to vary under such circum- 
stances as these is measured by its Standard Error (SE). In fewer than five adminis- 
trations in 100 will an item mean be more than two Standard Errors larger or smaller 
than its mean taken over all administrations. If you find the mean of an item on a given 
administration to be more than two SE from its overall mean, you can come to one of 
two conclusions: either (a) this is one of those relatively rare events (less than 5 in 100); 
or (b) something has happened in the school, and your assumption that conditions have 
remained the same is false. 

The Standard Error of an item mean can be estimated easily. To do so, you 
need to find several more numbers that are represented by _a through £ below: 

a. The total number of responses made to the questionnaire— that is, the 
sum of the Sfs of the individual items (SSf). 

b. The total value of all of those responses-ihat is, the sum of the Sfxs 
of the individual items (SSfx). 

c. The sum of the squared values of the responses— that is, the number 
shown as SSfx2, or more clearly symbolized as Sfi(l2) + lf2(2^) + 
2f3(32) + Sf4(42) + Sf5(52), where Sfj is the total number of "1** 
lesponses, Sf2 the total number of "2" responses and so on; this is 
calculated from column sums. 

d. The average number of responses per item— that is, Sti»p a_ dividec^ by 
the number of items, which is 18 in our brie? example (but actually woa^d 
be 70 in the staff questionnaire and 49 in the student questionnaire). 

The formula for the Standard Error uses the four numbers just defined and is shown in 
Table A-2. Table A-2 also shows, for the data of Tables A-1 and A-2, the four numbers 
defined above and the Standard Error.' 



will be noted that this approximate estimate of the Standard Error is based on the Total 
variance of the response matrix ra^,her than on the more precise Error (Respondents x Items) variance. 
Numerous analyses have made it clear that the approximate estimate is only slightly larger than the pre- 
cise estimate. The very small difference does not warrant the extra labor involved in computing the more 
precise estimate. 
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To use the Standard Error, you will need one more number: _ 

e. The overall meanH;hat is. Step b divided by Step a_ or X. This overall 
mean, the average response across all items, is the standard against 
which you can evaluate individual items. 

For the data summarized in Table A-2, the overall mean is 3.26 and the SE is 
.35. Take:i together, they indicate that items with means higher than 3.96, which is 
3.26 + (2 X .35), or lower than 2.56, which is 3.26 - (2 x .35) are probably not average 
items for this school and should be looked at as pointing at possible strong points and 
potential problems, respectively. 
5. Calculate TScores, 

For the lai^e array of item means obtained with an entire questionnaire, the 
inspection of item means is made simpler by transforming them into T-scores, which sets 
the mean equal to 50 and the Standard Error equal to 10. Nonaverage or "significant" 
items, then, are items with T-scores of 70 and above and 30 and below. 

The formula for T is given at the bottom of Table A-2: xj is the mean of a 
given item, £ is the overall mean of the data, and SE is the standard error as computed 
above. The last column of Table A-2 gives the T-scores for the items in the example. 

T-scores are almost indispensable if you have given both staff and student 
questionnaires in a single school or are interested in comparing data from different 
schools. You can make direct comparisons of item mean scores between staff and student 
questionnaires in the same school or between schools only when the overall means are 
the same, when the distributions of responses are equally variable and when the samples 
are of the same size. Experience indicates that none of these is likely to be the case in 
two questionnaire administrations. Consequently, if you want to compare scores from 
two questionnaire administrations, you need some way of compensating for all of these 
possible differences between one set of data and another. 

Table A-3 shows the itftm means ajid the corresponding T-scores obtained in 
two different schools on the first 12 items of the staff questionnaire. A direct comparison 



Table A-3 
Data From Two Schools 





Item Means 


T-Scores 


Item 


School A 


1 School B 


School A 


School B 


1 


3.50 


2.90 


51 


6 


2 


3.12 


3.14 


34 


18 


3 


2.58 


2.62 


9 


-9 


4 


3.81 


2.97 


65 


9 


5 


3.54 


3.50 


53 


37 


6 


3.50 


3.57 


51 


41 


7 


3.73 


3.15 


62 


19 


8 


3.3d 


3.00 


46 


11 


9 


4.31 


3.93 


88 


60 


10 


3.00 


2.73 


29 


-3 


11 


3.42 


3.38 


48 


31 


12 


3.46 


3.46 


49 


35 


N 


26 


30 






X 


3.48 


3.74 






SE- 

X 


.22 


.19 
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of the item means suggests that the problem of Item 2-The number of students who 
don't like going to school and don't do their school work-^ about the same m both 
schools. The T-scores, however, indicate that such students are rather more of a problem 
in School B. Similarly, Item 5-Capable students who feel that going to school ts pretty 
much a waste of time-would appear from the item means to be, if anything, a greater 
problem in School A, whereas the T-scores show it to be a rather greater problem m 
School B. And Item 12-Teoc/iero who won't admit making mistakes or thmk there is 
only one rioht answer to every question-gets the same mean score in both schools, 
although n is clearly more of a problem in School B. 

Our experience shows that student responses are almost uniformly the less favoraoie, 
that is, have lower overall means. Therefore, T-scores are probably, without exception, 
indispensable to detecting differences between staff and student responses to identical 
items in their questionnaires. 

6. Fill out the form. Summary of Staff and Student Responses (Figure A-2), finding 
mean scores for each section of the questionnaire. 

As noted earlier, each section of the questionnaire deals with a different aspect 
of the school. The mean T-score of the items in each of these sets gives some mdication 
of the aspect of the school which is most saUent in the eyes of the teachers, or of the 
students, as a sourct! of problems or of satisfaction. 

A form (Figure A-3) has rJso been provided for plotting the profile of a school 
as seen by teachers and as seen by students. This is a convenient way to visualize high 
points, low points, and differences between the staff and student perceptions. Data from 
one school in the pUot study have been transcribed onto the summary form (Figure A-2) 
and plotted on the profile form (Figure A-3) as an example. Mean T-scores larger than 100 
or less than 0 can be plotted as 100 and 0, respectively, without seriously affecting 
the profile. 
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School Problem Area Surveys: 
Sample Summary of Staff and Student Responses 
School £iO j^^^ 



s 


taff 


StudenyH^Jj^ Staff 


Student 


Staff 


Student 


Item 


T 


Item 


T 


item 


T 


Item 


T 


Item 


T 


Item 
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^ 
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43 
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DO 
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NOTES ON PUNCHING CARDS AND RUNNING PROGRAM FOR 
ANALYSIS OF STAFF AND STUDENT DATA 



Running the Staff or Student Tab Programs 

Both programs are written in FORTRAN IV. The data pack for either program 
consists of the appropriate cards, one per questionnaire (respondent), preceded by one 
special card. The special card has the school identifying number in columns one and two. 
It also has the total number of questionnaire cards (respondents) in columns three 
through five. The name of the school, if wanted in the output, can be listed in columns 6 
through 25 of the special card. 

Keypunching Staff Questionnaires for 

input to the Staff Tab and Statistical Program 

The cards used as input to this program are read by means of the following FORTRAN 
format statement: 

F?)RMAT(I2, I3,(1X,18I1), 1X,14I1,1X,9I1,1X,11I1) 

A data card punched according to this format is shown in Figure B-1. The five-digit 
number in columns 1-5 is an identifier. The first two digits identify the school; the next 
three digits identify the individual questionnaire. Responses are grouped on the card as 
are the questions on the questionnaire. 

Sample Punched Card: Staff Questionnaire 



oellnoiiioiiNiMQiNaooeiaoooooocoooooooooooooooooooooooooooooooocoooooooooooo 
iiniilniilniiliiiilntnlinilnMliMiliniilinilMiitliHiliiiMliinliiM 

2l22l22l2222l2222l2222l22 222l2 2 22l2 222l2 22 2 2l2 2 2 2l2 22 2l2 2 2 22l2 2 22l2 222 2l22 2 2l2 22 
33 3 33 3 3313 33 313 33313 33313 333 3133 33l3333l33333l3333l3333l33 3331333 3l33 3 33I3333I33 
44 4 44 444 (14 (4 (|( (4414 (44 414 44414 4 4414 44414 44 4 414 4 4 414 4 4 414 44 4 414 4 4 414 4 44 414 4 4 414 

lSSS$5 5SS5lSSSSlS5SSlS5SSSlS555l5555l5S55l5S5 55lS5 55l55 55l55b5 5lSS555l555 5lS5 55l 
6 (fi(((fi(((fifi6fifififififififififififififififififi(fi6(fifi6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6fifi 66666 set 66 fi666fiMfi6 6 6 6 6 6 6 

7 7 7 7 n 7 7 7 7 7 7 7 7 7 7 // 7 ////n 7 7 7 7 7 7 7 7 7 7 7 7 //// ///n 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 // n 7 7 7 7 7 7 7 7 / 

mil mil niiiii8i8SM8iii aisMsiinisisMSs 

9999999999 999S99999999999 9999999 9999 9999 99999 99999 9999999999S9999 99999 999S9 99999 



Figure B-1 
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Responses are punched as the number in the leftmost box marked by th^ 
respondent. The boxes are numbered from 1 to 6 starting from the left.' If there is no 
response, a digit outside this range should be punched. 

With the use of a "set-up" card, a staff questionnaire can be keypunched in about 
60 seconds. 



Keypunching Student Questionnaires for 

Input to the Student Tab and Statistical Program 

The cards used as input to this program are read by means of the following FORTRAN 
format statement: 

F?)RMAT(I2,I3,IX,20I1,1X,9I,1X,11I1,1X,2I1,1X,5I1,1X,2I1) 

A data card punched according to this format is shown in Figure B-2. The five-digit 
number in columns 1-5 is an identifier. The first two digits identify the school, the next 
three digits identify the individual questionnaire. Responses are grouped on the card as 
are the questions on the questionnaire. 

Sample Punched Card: Student Questionnaire 



iilliiiiosi 1011 mil Mioiaina 000 0 0 00 0 00 0 0 0 0 ooono 0 0 0 0 0 0 0 000 0 00 0 0 0 0 0 0 0 0 ( 
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Figure 8-2 



Responses are punched as the number in the leftmost box marked by the student. 
The boxes are numbered from 1 to 6 starting from the left. If there is no response, a 
digit outside this range should be punched. 

With the use of a **set-up'* card, a student questionnaire can be keypunched in 
about 40 seconds. 



'The value "6" for * 'exceptionally good" responses is converted by the computer program to a 
value of "5" for computing means and standard errors. A simple frequency count of "exceptionally good" 
responses is obtained for each questionnaire item and appears in the computer printout. 
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Appendix B-2 



FORTRAN PROGRAMS FOR ANALYZING DATA FROM 
STAFF AND STUDENT QUESTIONNAIRES 

Each program is followed by a specimen printout of data. 



Sample Program: Staff Data 

C STAFF TAB AND STATISTICAL PROGRAM 

1 DIMENSION NMSCHL(5)JTAB(70,7),NR(70),ITSUM(7),INTS(3)^M(5), 
1QA(70,2),KSM(2),N6P(12)JMN(12) 

2 DATA NGP/0,6,6,6,6,6,6,14,9,7,2,2/ 

3 101 FORMAT(IHI) 

4 150 F0RMAT(I2,I3,2(1X,18I1),1X,14I1,1X,9I1,1X,11I1) 

5 200 FORMAT(5X,2HQ#,4X,40H EXT PROS CONSRBLE MODERATE LITTLE 

150H NOPROB EXCGOOD NOANS MEAN T ) 

6 300 F0RMAT(1H1,5A4) 

7 400 FORMAT(6X,10HSCHL ID # J2,5X,11H# OF STAFF ,13) 

8 500 FORMAT(IX) 

9 600 FORMAT(1HO,6HTOTALS,1X7I10,1F10.2,1F10.3) 

10 700 FORMAT(1XJ6,1H.7I10,1F12.2,1F10.0) 

11 900 FORMAT(5X,5HSIGMAJF10.2) 

12 REAC(5J00)NSCHL,NKARD,NMSCHL 

13 100 F0RMAT(I2J3,5A4) 

C FIRST INITIALIZE ARRAY & CARD COUNTER 

14 DO 11 1=1,70 

15 DO 10J=1,7 

16 ITAB(;,J)=0 

17 10 CONTINUE 

18 11 CONTINUE 

19 KARD =0 

C INPUT AND TABULATE RESPONSES 

20 1 READ(5,150)NS,NPUP,(NR(I),I=1,70) 

21 DO 12 1=1,70 

22 IF (NR(l).Ea.0.DR.NR(l).GT.6)NR(l)=7 

23 ICOL=NR(l) 

24 ITAB(I,IC0L)=ITAB(I,IC0L)+1 

25 12 CONTINUE 

26 KARD=KARD+1 

27 IF(KARD.LT.NKARD)GO TO 1 
C CALC TABSUMS 

28 D0 21J=1,7 

29 ITSUM(J)=0 

30 21 CONTINUE 

(Continued) 
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ERRATA' 



Type of 

Error 


State- 
ment 
Number 


Correct Version 






oampie rrogram: btatt uata (pagd oO) 


Spacing, 
typing 


5 


200 FORMAT(5X,2HQ#,4X,40H EXTPROB CONSRBLE MODERATE LITTLE, 
150H NOPROB EXCGOOD NOANS MEAN T ) 


typing 


12 


READ(5,100)NSCHL,NKARD,NMSCHL 


typing 


22 


IF (NR(l).EQ.0yOR.NR(l).GT.6)NR(l)=7 


typing 


51 


SIGG=SQRT((SM(3)-(SM(2)**2)/SM(1))/(SM(1)~1.)) 
C CALC MEAN,SIGMA,T FOR EACH QUESTION 


typing 


70 


IF=II+NGP(J)~1 


typing 


71 


TMN(J)=AVGT(II,IF,QA) 


spacing 


89 


1100 FORMAT(55H0 SA SS TS TT PS PT SC SP, 



128H CP AP FP TP) 



Sample Program: Student Data (page 64) 

typing 1 DIMENSION NMSCHL(5),ITAB(49,7),NR(49),ITSUM(7),INTS(3),SM(5), 

1Q/^,I9,2),KSM(2),NGP(12),TMN(12) 

typing 3 150 FORMAT(I2,I3,1X,20I1.1X,9I1,1X,1 1I1,1X,2I1,1X,5I1,1X,2I1) 

spacing 4 200 FORMAT(5X,2HQ#,4X,40H EXTPROB CONSRBLE MODERATE LITTLE , 

150H NOPROB EXCGOOD NOANS MEAN T ) 

typing 22 ICOL^'NRd) 

indention [34] 22 CONTINUE (Statement 34 is repeated for Indention Guidance) 



35-41 C CALC INTERMEDIATE SUMS 

DO 31 1=1,3 (35) 

INTS(l)=0 (36) 

31 CONTINUE (37) 

DO 32 J= 1,5 (^) 

INTS(1)=INTS(1)+ITSU:.MJ) (X) 

INTS(2)=INTS(2)+J*ITSUM(J) (40) 

INTS(3)=INTS(3)+(J**2)*ITSUM(J) (41) 



typing 64 QA(I,2)=50.+ 10 *(QA(I,1 )-RMEAN)*SQRT(SM(4))/SIGG 



spacing, 89 1100 FORMAT(55H0 SA SS TS TT PS PT SC SP, 
typing 128H CP AP FP TP) 

^1^1^" * These errata are printed on gummed paper and can beg^ut and gtued over the original statements, if desired. 



31 DO 22 J=17 

32 DO 23 1=1,70 

33 ITSUM(J)=ITSUM(J)+ITABtl,J) 

34 23 CONTINUE 

35 22 CONTINUE 

C CALC INTERMEDIATE SUMS 

36 DO 31 1=1,3 

37 INTS(l)=0 

38 31 CONTINUE 

39 DO 32 J=l,5 

40 INTSd )=INTS(1 )+ITSUM(J) 

41 INTS(2)=INTS(2)+J*ITSUM(J) 

42 IN1S(3)=INTS(3)+(J**2)*ITSUM(J) 

43 32 CONTINUE 

44 INTSd )=INTS(1 )+ITSUM(6) 

45 INTS(2)=INTS(2)46*ITSUM(6) 

46 INTS(3)=INTS(3)+25*ITSUM(6) 

47 DO 33 1=1,3 

48 SM(I)=INTS(I) 

49 33 CONTINUE 

C CALC MEAN & SIGMA FOR EACH GROUP 

50 RMEAN=SM(2)/SM(1) 

51 SIGG=SQRT((SM(3)-(SM(2)**2)/SM(1)/(SM(1)-1.)) 
C CALC MEAN, SIGMA, T FOR EACH QUESTION 

52 DO 51 1=1,70 

53 DO 52 K=l^ 

54 KSM(K)=0 

55 52 CONTINUE 

56 DO 53 K=l,5 

57 KSM( 1 )=KSM(1 )+ITAB(l,K) 

58 KSM(2)=KSM(2)+K*ITAB(I,K) 

59 53 CONTINUE 

60 KSMd )=KSM(1 )+ITAB(l,6) 

61 KSM(2)=KSM(2)+5*ITAB(I,6) 

62 . SM(4)=KSM(1) 

63 SM(5)=KSM(2) 

64 QA(l,t)=SM(5)/SM(4) 

65 )QA(I,2)=50.+10.*(QA(I,1)-RMEAN)*SQRT(SM(4))/SIGG 

66 51 CONTINUE 

C CALCULATE GROUP MEANS 

67 IF=0 

C8 DO 905 J=l,12 

69 ll=IF+l 

70 IF=II+NGF(J)-1 

71 IMN(J)=AVGT(II,IF,QA) 

72 905 CONTINUE 

73 TMN(1)=100000. 
C PRINT OUTPUT 

74 WRITE(6,300)NMSCHL 

75 WRITE(6,400)NSCHL,KARD 



(Continued)^ 
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76 WRITE(6,200) 

77 WRITE(6,500) 

78 DO 61 1=1,70 

79 WRITE(6,700)I,(ITAB(I,J),J=1,7),(QA(I,K),K=1,2) 

80 IF(I.EQ.6.OR.I.EQ.12.OR.I.EQ.18.OR.I.EQ.24.OR.I.EQ.30.OR.I.EQ.36) 
1WRITE(6,500) 

81 IF<I.EQ.50)WRITE(6,101) 

82 IF(I.EQ.59.OR.I.EQ.66.OR.I.EQ.68)WRITE(6,500) 

83 61 CONTINUE 

84 WRITE<6,600)(ITSUM(I),I=1,7),RMEAN 

85 WRITE(6,900)SIGG 

. C PRINT GROUP MEANS 

86 WRITE<6,1100) 

87 WRITE(6,1000;rMN 

88 1000 FORMAT(1HO,12F7.1) 

89 1100 FORMAT(55H0 SA SS TS TT PS PT SC SP, 

128H CP AP FP TP) 

90 STOP 

91 END 

92 FUNCTION AVGT(II,IF,QA) 

93 DIMENSION QA(70,2) 

94 SUMT=0. 

95 DEN=IF-II+1 

96 IF(DEN.GT.0.)GO TO 934 

97 AVGT=0. 

98 RETURN 

99 934 DO 935 NIIJF 

100 SUMT=SUMT+QA(I,2) 

101 935 CONTINUE 

102 AVGT=SUMT/DEN 

103 RETURN 

104 END 
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Sample Printout: Staft Data 



AGUAJITO Jft HIGH 
SCHL 10 4 ^3 
Q« EXT PRCS 



1. 
2. 
3. 
^. 

5 
6 

7. 

8. 

9. 
10. 
11. 
12. 

13. 

\t: 

16. 
17. 
18. 

19. 
20. 
2U 
22. 
23. 
2^. 

23. 
26. 
27. 
28. 
29. 
30. 

iU 
32. 
33. 
3^. 
33. 
36. 

37. 
38. 
39. 
<iO. 
^1. 

ti: 

^3. 
^6. 
^7. 
^8. 
^9. 
50. 
31. 
32. 
33. 
5^. 
33. 
36. 
37. 
38. 
3 . 

60. 
61. 
62. 
63. 
6^. 
63. 
66. 

67. 
68. 

69. 
70. 

TOTALS 

SIGMA 



0 
0 
11 
5 
6 
1 

0 
0 
0 

1 

2 
0 

0 
2 

0 
0 

0 

8 
9 
1 

3 

0 
2 
1 
0 
3 
1 

0 
1 
1 

0 
3 
3 

3 
19 
12 
2 
6 
7 
7 
2 
0 
1 



2 
0 
0 
^ 
0 

I 
I 

3 
9 
I 

7 
2 
1 
7 
10 
*0 
7 



« OF STAFF 
CONSRELE ^ 

12 
13 
26 
21 
16 
3 

1 

12 
3 

1 

2 

J 
3 
9 
7 
4 

0 

2 

8 
13 

3 
13 

1 
5 
9 
3 
6 
3 

3 
IC 

3 

8 
11 
13 

11 
13 
19 
17 

13 
13 
16 

9 

3 

3 
Id 
10 

3 

4 

C 

3 

2 
Id 
13 

'S 

t! 

11 



0 
2 

2U 
1.13 



SA 



2 
IC 

3d0 



TT 



69.^ 



SS TS 
11.8 67.8 
CORE USAGfc 03JECT Cf'OE* 
COHPILE T1MF.« 0.36 



PS 



o 

CD A T C 
CK A 1 C 


1 1 T Ti Ik 


n J r f\ uo 


28 


io 


4 






c 


12 


a 


1 
1 


20 


9 




23 


0 




20 






16 


18 




12 


22 


0 


1 7 


ia 


0 


1 1 
i 1 






16 


24 




2i 


22 


# 


13 


30 


6 




22 


» 
f 


13 


ii 


0 

TT 


13 


22 




16 


22 


1 £ 


8 


17 


20 


5 


20 


18 


8 


1 1 


1 7 

i r 


16 


IS 
1 3 


1 


14^ 


1 1 


7 


13 


1 ^ 
}? 


1 3 


1 3 


1 i 


1 <\ 

1 V 


8 


13 


20 


16 


1 0 
19 




12 


1 r 


1 u 




14 


O K 
C 9 


9 


9 


1 f 


3 


16 


1 Q 


21 


21 


1 u 




^2 


9 


8 


0 


28 


7 


1 7 




11 


18 


10 


13 


14 


8 


21 




c 


1 8 


17 




16 




u 


"> n 
cu 


0 


n 

u 


2 1 




£ 


24 






2 7 


1 n 
1 U 


1 
i 


19 


1 1 
1 1 


1 


21 


1 7 
1 r 


\ 


17 


21 


5 


7 


33 


8 


20 


£ f 




18 


1 n 
1 u 




Cc 


1 Q 




12 


22 


1 7 
1 f 


6 


16 


29 


28 


1 1 


0 


13 


29 


13 


11 


24 


16 


3 


20 


23 


6 


22 


21 


21 


7 


2 


17 


10 


1 n 
1 0 


20 


11 


2 


1 


11 


29 


JL7 


17 


1 3 




1 1 


3 


1 ^ 

{2 


1 1 

16 


3 
\l 


12 


1 "X 
1 3 


1 ; 


3 
8 


^? 


M 


9 


9 


21 


20 


10 


4 


1038 


1168 


717 




PT SC 


SP 


• 6 


84.3 64.7 


19.6 



EXC GCOC 



3 
8 

14 

3 
2 
1 
4 
1 

14 
1 

8 

3 
3 
9 

2 
13 
7 
3 
2 
1 

0 
0 

8 

0 
0 
0 
0 
0 
0 
2 
0 
0 
0 
0 
0 
0 
0 
4 
1 
2 
0 
0 

1 

9 
2 
2 



1 

2 

10 
4 

178 



NO ANS 

? 

0 
0 
0 
0 

4 
0 

? 



4 
0 
0 
4 
6 
12 
5 
4 
1 
1 
3 
4 
2 
1 

I 

2 
1 
7 

1 
4 
3 
3 
3 
2 
3 

9 
4 

7 
8 

168 



MEAN 

3.1^ 
2.98 
2.34 
2.72 
2.81 
3.50 

4.02 
3.59 
3.69 
3.84 

3-82 
3.70 

3.84 
3.66 

1:51 

3.84 
'-.21 

4.47 
3.96 
3.16 
2.89 
3«84 
3.18 

4.41 
3.70 
3.74 
4.23 
3.84 
4.30 

I'M 
X:ll 

3.43 
3.20 

3.12 
3.07 
2.09 
2.42 
3.39 
2.B6 
2.74 
2.69 
3.19 
3.61 
3.92 
3.44 
2.73 
3.23 
3.93 
4.31 
2.82 
3.93 

X'.W 

4.07 
2.36 
3.29 

2.77 
4,30 
3.73 
2.91 
2.60 
3.09 
3.22 

4.31 
4.15 

4.33 
3.24 

3.30 



28. 
16. 
-26. 
-1. 

4. 
30. 

83. 

36. 
62. 
72. 
71. 
63. 

72. 
60. 
32. 
64. 
72. 
96. 

113. 
80. 
28. 
10. 
72. 
29. 

110. 
63. 
63. 
99. 
70. 
99. 

63. 
63. 

21: 

47. 
30. 

23. 

22. 
-43. 
-21. 

43. 
8. 

-0. 

-2. 

31. 

36. 

77. 

46. 
1. 

32. 

78. 
102. 
6. 

78. 

89. 

89. 

87. 
-12. 

37. 

2. 
113. 
64. 
12. 
-8. 
23. 
32. 

99. 
91. 

103. 
34. 



CP 

61.6 



AP 



FP 



TP 



34.1 94.9 68.3 



3160 8YTeS,ARRAy ARE*' 
SECfEXECUTION TIME* 3.22 SEC, 



2988 8YTES,T0TAL AREA AVA1LA8LE« 8288 BYTES 
nATFIV - VERSION 1 LEVEL 2 AUGUST 1970 



Sample Program: Student Data 



C STUDENT ^AB AND STATISTICAL PROGRAM 

1 DIM-..*' )ION NMSCHL(5),ITAB(497),NR(49)JTSUM(7)JNTS(3),SM(5), 
1QA( . ),KSM(2),NGP(12),NGP(12)TMN(12) 

2 DATA NGP/6,6,6,3,6,a2 J 1 ,2,5,2,0/ 

3 150FORMAT(12,I3,1X,20I1,1X,9I1,1X,11I1,1X,2I1,1X,5I1,1X,2M) 

4 200 FORMAT(5X,2HQ#,4X,40H EXT PROB CONSRBLE MODERATE LITTLE, 

150H NO PROB EXC GOOD NO ANS MEAN T) 

5 300FORMAT(1H1,5A4) 

6 400rOF:MAT(5X,10HSCHL ID # ,I2,5X,14H# OF STUDENTS ,13) 

7 500FORMAT(1X) 

8 600FORMAT(1HO,6HTOTALS,1X,7I10,1F10.2,1F10.3) 

9 700 FORMAT(1XJ6,1H.,7I10,1F12.2,1F10.0) 

10 900 FORMAT(5X,5HSIGMA,1F10.2) 

11 READ(5,100)NSCHL,NKARD,NMSCHL 

12 100FORMAT(I2,I3,5A4) 

C FIRST INITIALIZE ARRAY & CARD COUNTER 

13 DO 11 1=1,49 

14 DO10J=l,7 

15 ITAB(l,J)=0 

16 10 CONTINUE 

17 11 CONTINUE 

18 KmRD=0 

C INPUT AND TABULATE RESPONSES 

19 1 READ(5,150)NS,NPUP,(NR(I),I=1,49) 

20 DO 12 1-1,49 

21 IF(NR(l).EQ.0.OR.NR(l).GT.6)NR(l)=7 

22 ICCL=NR(I) 

23 ITAB(l,ICOL)=rMS(l,ICOL)+l 

24 12 CONTINUE 

25 KARD=KARD+1 

26 IF(KARD.LT.NKARD)GO TO 1 
C CALC TABSUMS 

27 D0 21J=1,7 

28 ITSUM(J)=0 

29 21 CONTINUE 

30 DO 22 -1,7 

31 DO 23 1=1,49 

32 ITSUM(J)=ITSUM(J}+ITAB(I,J) 

33 23 CONTINUE 

34 22 CONTINUE 

C C LC INTERMEDIATE SUMS 

35 00 311=1,3 

36 INTS(l)=0 

37 31 CONTINUE 

38 D0 32J=1,5 

39 INTS(1)=INTS(1)+ITSUM(J) 

40 INTS(2)=INTS(2)+J*ITSUM(J) 

41 INTS|3)=»INTS(3)+J**2)*IT3UM(J) 

(Continued) 
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•51 

?0 



42 




32 CONTINUE 


43 




INTS(1)=INTS(1)+ITSUM(6) 


44 




INTS(2)=INTS(2)+5*ITSUM(6) 


45 




INTS(3)=INTS(3)+25*ITSUM(6) 


46 




DO 33 1=1,3 


47 




SM(I)=INTS(I) 


48 




33 CONTINUE 




C 


CALC MEAN & SIGMA FOR EACH GROUP 


49 




RMEAN=SM(2)/SM(1) 


50 




SIGG=SGRT((SM(3)-(SM(2)**2)/SM(1))/(SM(1)-1.)) 




C 


CALC MEAN, SIGMA, T FOR EACH QUESTION 


51 




DO 51 1=1,49 


52 




DO 52 K=1,2 


53 




KSM(K)=0 


54 




52 CONTINUE 


55 




DO 53 K=1,5 


56 




KSM(1)=KSM(1)+ITA8(I,K) 


57 




KSM(2hKSM(2)+K*ITAB(l,K) 


58 




53 CONTINUE 


59 




KSM(1)=KSM(1)+ITAB(I,6) 


60 




KSM(2)=KSM(2)+5*ITAB(I,6) 


61 




SM(4HKSM(1) 


62 




SM(5hKSM(2) 


63 




QA(l,1hSM(5)/SM(4) 


64 




QA(1,2)=50.+10.*(QA(l,1)-RMEANi*SQRT(SM(4))/SlGG 


65 




51 CONTINUE 




C 


CALCULATE GROUP MEANS 


66 




IF=0. 


67 




DO 905 J=1,12 


68 




INIF+1 


69 




IF=II+NGP(J)-1 


70 




TMN(J)=AVGT(II,.F,QA) 


71 




905 CONTINUE 


72 




TMN(6)-100000. 


73 




TMN(12)=100000. 




C 


PRINT OUTPUT 


74 




WRITE(6,300)NMSCHL 


75 




WRITE(6,400)NSCHL,KARD 


76 




WRITE(6,200) 


77 




WRITE(6,500) 


78 




DO 61 1-1,49 


79 




WRITE(6,700)I,(ITAB(I,J),J=1,7),(QA(I,K),K-1,2) 



iV) IF(LEQ.6.OR.I.EQ.12.OR.I.Eai8.ORJ.EQ.21)WRITE(6,500) 

81 IF(I.EQ.27.OR.LEQ.29.OR.I.EQ.40.OR.LEQ.42)WRITE(6,500) 

82 IF|I.EQ.47)WRITE(6,500) 

83 61 CONTINUE 

84 WRITE(6,600)(ITSUM(l),l=1,7),RMEAN 

85 WRITE(6,900)SIGG 

C PRINT GROUP MEANS 

86 WRIT=(6,1100) 

{Continued) 



71 



87 WRITE(6,1000)TMN 

88 1000 FORMAT(1HO,12F7.1) 

89 1100FORMAT(55H0 SA SS TS 

128H CP AP FP TP) 

90 STOP 

91 END 

92 FUNCTION AVGT(II,IF,QA) 

93 DIMENSION QA(49,2) 

94 SUMT=0. 

95 DEN=IF-II+1 

96 IF(DEN.GT.0.)GO TO 934 

97 AVGT=0. 

98 RETURN 

99 934 D0935 l=ll,IF 

100 SUMT=SUMT+QA(I,2) 

101 935 CONTINUE 

102 AVGT=SUMT/DEN 

103 RETURN 

104 END 
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Sample Printout: Student Data 



PALO COLORADO SCHOOL 



SCHL in t 3^ 



Q« 

I. 
2. 
3. 
4. 
5. 
f*. 

7. 

9. 
in. 
II. 
I?. 

13. 
lA. 
ih. 
16. 
17. 

2(1. 
?l. 

23. 
2^. 

. 

27. 

28. 
29. 

30. 
?1. 
32. 
^3. 
^4. 
35. 
3f . 
37. 
?H. 
39. 
^O. 

M. 

^7. 

4? . 

^S. 
^o. 
^7. 

TOTAL S 

SI CM* 

<;a SS 

S6.n 49 

COR? USAGE 



EXT PPOB 

? 
7 
3 

3 
I 

2 

2 

5 
4 



7 
2 
I 

2 

e 

10 

. o 

*6 
6 

10 

I? 
i 
( 
8 

II 



4 

5 

?h7 
t. 



* OP STUDENTS 
CONSRBLt ^'OtM 

3 

V 

9 
7 
2 
f 

S 

*5 
2 
6 
9 
3 

14 
13 

2 

n 

7 



6 

/; 
/ 

7 
7 



T<; 
47. 



3 

12 
/> 
S 
7 

? 
10 



S 
3 
6 
6 



•^7. I 



56 



► n« 1 c 


LI TTLE 


NO PROB 


fXC 


GOOD 


NO AN 


10 


19 


10 




4 


0 




i4 


9 






0 




14 


3 




3 


0 


9 


?2 


4 




0 


0 


5 


9 


22 




7 


0 


7 


17 


10 






0 


1 0 


12 


9 




10 


0 


15 


1 1 


5 




4 


I 


1 1 


21 


9 




3 


0 


13 


19 






I 


0 


12 




I 




3 


0 




fc t 


r 




2 


I 


7 


15 


8 




3 




9 


13 


3 




4 






[ ) 


14 




I 


1 




1 4 


3 




3 


I 


8 


1 c 

I ^ 


9 




5 


■ 


A 

•» 


1 ? 


1 o 




3 




1 


1 I 


21 




10 


1 


T 

r 




I 5 




g 


1 






4 




2 


1 


3 


7 


13 




21 


I 


13 


12 


9 




3 


? 


1 « 




7 




2 




14 


13 






2 


0 


11 


10 


15 




3 






1 Q 
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Appendix C 

NOTES ON DEVELOPMENT OF THE INSTRUMENTS 



The Questionnaires 

Tfie questionnaires were constructed from a pool of 300 general items for staff and 
200 general items for students. Since we were concerned with screening for problems, we 
were interested in items that could be expected to have generality across schools. While a 
pool containing many more specific items could have been developed, their level of detail 
and questionable generality across many schools would have made them of 
doubtful utility. 

The preliminary questionnaires used in the main pilot study were scored by 
computing the percentage giving favorable responses and the percentage giving 
unfavorable responses to each item. In the final form of the questionnaires, scores are 
computed by weighting responses on a scale from 1 to 5 and obtaining item means. While 
mean values are more efficient and use more of the information, we found them to be 
highly related (r = .98) to percentage scores. 

The items were tested in one major and several minor pilot studies. Written (and in 
some schools, verbal) comments provided by respondents were useful in identifying 
ambiguous items or items that attempted to elicit information not typically available to 
the respondents. Highly correlated items were identified by standard correlational 
techniques and redundant items were eliminated. 

Conversion of item mean^; to T-scores helped identify items that were consistently 
high or low in most schools, since the conversion eliminates the variations in the general 
level of response found in data from different schools. Items that elicit strong consensus 
across schools suggest stereotype reactions which may have little utility for diagnostic 
purposes. For example. The teachers usually try to relate to their students in a positive 
and accepting manner was significantly favorable in 24 out of 33 schools. At the other 
extreme, another item. Non-teaching activities take up an excessive amount of the 
teachers* time, was significantly unfavorable in 17 of the 33 schools. 

To explore this problem further, we combined the data of 33 schools, treating all 
responses as though they had come from one huge school. We then found an overall 
T-score for each item. However, because of a considerable discrepancy in numbers of 
responrlants from school to school, overall T-scores are not directly comparable to the 
T-scores found in individual schools. To make them comparable, we computed the 
expected T-score for each item in each school. Expected T-scores are based on the overall 
item scores but use the number of respondents appropriate to each school in computing 
the Standard Error. They show, in effect, for each school, what the T-scores would be in 
an average school of tlie same size. Differences between observed and expected T-scores 
show how much the school deviates from the average school in reject to each item. 
Such differences are independent of both the overall score (the response level) in each 
school and of the overall score (the req)onse level) of each item. 

The comparisons between observed and expected T-scores were informative. For 
example, the first item mentioned above. The teachers usually try to relate to their 
students in a positive and accepting manner (which was significantly favorable in 24 out 
of 33 schools), showed little deviation from the expected values in any school. Although 
the response to the item was uniformly favorable, it was at the same relative level in all 
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schools. Clearly, the item is a clich^. It elicits a uniformly favorable stereotype in all 
schools, but it does not discriminate among schools. While it may represent a strength in 
all schools, it is lypparently not a strength unique to any one school or set of schools. 
However, the second item mentioned above. Non-teaching activities take up an excessive 
amount of the teachers' time (which was significantly unfavorable in 17 out of 
33 schools), differed considerably from expectation in 11 schools, unfavorably in 5 and 
favorably in 6. WhOe this second item elicits an unfavorable stereotype in most schools, it 
does discriminate among schools. An apparently unfavorable response made in a given 
school may, in fact, be favorable when it is compared to the response made in the 
average school. 

Based upon this analysis, items that clearly r<)flected no discrimination among 
schools were eliminated. 

Implications of the Analysis of Observed and Expected T-scores. The term 
""average," in the analysis of Observed vs. Expected T-scores, refers to the schools in our 
sample. Our data cannot be generalized beyond our specific sample because we did not 
employ a definable random sample of schools, but rather, a sample of convenience, based 
upon the availability of research staff members in three geographic locations across the 
country, who established access to junior high schools for data collection. We do believe, 
however, that data from a truly random sample of junior high schools would not differ sub- 
stantially from what we now have. 

WhUe Observed T-scores (for the various questionnaire items) are valid 
indicators of the relative intensliy of req)onse in a school, in planning prc^ams or 
procedures to deal with problems in a school it would be useful to know how teachers or ^ 
students in general react to a given item. Doing something to relieve teachers of some' of 
the non-teaching demands on their time, for example, is likely to have a salutary effect 
in ahnost any school. The question is: Are there problems more nearly unique to this 
school that are more deserving of attention? Part of the answer lies in the differences 
between Observed and Expected T-scores. 

Although general or national norms are not presently available for the 
questionnaires, unrestricted statistical generality is not wholly necessary for expected T*s 
to have practical utUity. We do have data avaUable that suggest that reasonable 
evaluations can be made of responses in a school if data are available from other schools 
in the district. This evidence comes from part of our pilot work in which we collected 
questionnaire data in aH 11 junior high schools in one large mid western school district. 
Analysis of these data indicated that virtually the entire range of problems covered by 
the questionnaires appeared in the 11 schools in this one district. 

Therefore, in the absence of national norms, we suggest that, whenever 
possible, all of the schools in a system be tested at about the same time. If the system is 
of fair size. Expected T-scores can be based on the pooled data from the whole system, 
and the problems relatively unique to each of the individual schools can be identified. 

The Interview Form 

Preliminary versions of the interview form were analyzed to identify 
nondiscriminating and redundant items. As an example of nondiscriminating items, one 
designed to determine whether a principal felt he was given enough autonomy, elicited 
only one negative rei5)onse among 34 principals. Items of this sort, which yielded 
essentially no information, were eliminated. 

To identify redundant items, we examined interrelationships among variables using 
conventional chi-square techniques. To do this, we first identified all items for which 
discrete categories were predefined for recording req)onses or which lent themselves to 
construction of discrete response categories following data collection. We then split each 
item at the median. For mc .t variables, the approximate median provided a reasonable 
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division. In 17 of 34 schools, for example, less than 20% of the students were on a free 
lunch program, and in the other 17 more than 20% were. Only an approximate m'xiian 
could be established for some varia'ies. For example, in 16 of these schools, all of the 
teachers were white, and in the remaining 18 schools there was at least one nonwhite 
teacher. While this is conveniently close to a median spli^, there is no assurance that 
all-white vs. not-all-white divides the variable in the most meaningful way. In all, 
37 usable variables were identified and put into a fourfold table with each of the other 
36 for analysis by chi square.^ 

The intercorrelations among these 37 variables revealed some clustering, though none 
of the intercorrelatiuns was so high as to suggest that some of the variables might be 
redundant. One variable. Structure, which divided the schools into two classes, exclusively 
Junior High (or Middle) Schools with students in only 6, 7, 8, or 9th grades, and Others, 
where other grades were present, showed a clear pattern of relationship with many of the 
other variables. On the whole, dividing tt^is set of schools into Junior High Schools and 
Others appears to divide their districts essentially into two sorts: affluent, surburban, and 
predominantly middle class vs. poor, urban, and not predominantly middle class. In 
general, the Other schools had the more problems, with more students on probation, 
more suspensions, more pregnancies, more disciplinary problems and the like. The 
exclusively Junior High Schools, however, were more often noncity schools, had higher 
per pupil expenditures, offered more services, and had lower or nonexistent drug abuse 
and veneral disease rates. 



'These analyses were not undertaken as a formal study of relationships. With more variables than 
schools, such an intent would be statistically mdefensible. The analyses, however, we^o of heuristic value 
in the early stage of development. 
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The f St step in the problem-solving process is to become aware that problems 
exist. The second is to identify the problems with some degree of precision- 
both in terms of their "real-world" aspects and of the ways in which they 
are perceived by the individuals and groups involved. 

This manual presents three easy-to-use instruments that school 
personnel and mental health consultants can use to conduct 
systematic explorations of those features of a school and 
Its setting that may give rise to individual, group, and/or 
systemic problems. It also describes procedures to be 
used in analyzing and interpreting the data col- 
lected with these instruments. 



Both the instruments and the procedures 
are simple to use. They proved to be 
useful, reliable, and valid, when the 
authors pilot-tested their new 
approach in 40 schools In 
six states. 
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